2023-10-06T01:17:02.226 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T01:17:02.227 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088 --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/orig.config.yaml 2023-10-06T01:17:02.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T01:17:02.347 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} email: yweinste@redhat.com first_in_suite: false job_id: '7412088' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPjxjWApRZqgOMmRkqAKtxkRzr0GOGEOY3Hf3J2iWiMgMQJMtCLfNCMwNhkeTMl99ekKWHz1qTVHUFzssA54Bc= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI00sWvHkKzqXh1e1/v1VTW2/GSd8VRipyNfxlH5V5o99+A1owTp6qbpism66RZ736kOpXoRoX8tHFblaGu4KLs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T01:17:02.347 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T01:17:02.348 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T01:17:02.348 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T01:17:02.350 INFO:teuthology.task.internal:Checking packages... 2023-10-06T01:17:02.372 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T01:17:02.372 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T01:17:02.373 INFO:teuthology.packaging:ref: None 2023-10-06T01:17:02.373 INFO:teuthology.packaging:tag: None 2023-10-06T01:17:02.373 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T01:17:02.373 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T01:17:02.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T01:17:02.593 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296.g736ac5ba 2023-10-06T01:17:02.593 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T01:17:02.600 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T01:17:02.600 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T01:17:02.615 INFO:teuthology.task.internal:Saving configuration 2023-10-06T01:17:02.632 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T01:17:02.637 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T01:17:02.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:12:01.034911', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPjxjWApRZqgOMmRkqAKtxkRzr0GOGEOY3Hf3J2iWiMgMQJMtCLfNCMwNhkeTMl99ekKWHz1qTVHUFzssA54Bc='} 2023-10-06T01:17:02.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:12:01.036913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI00sWvHkKzqXh1e1/v1VTW2/GSd8VRipyNfxlH5V5o99+A1owTp6qbpism66RZ736kOpXoRoX8tHFblaGu4KLs='} 2023-10-06T01:17:02.685 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T01:17:02.692 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T01:17:02.692 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T01:17:02.692 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T01:17:02.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f50002aeb80>, signals=[15]) 2023-10-06T01:17:02.761 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T01:17:02.767 INFO:teuthology.task.internal:Opening connections... 2023-10-06T01:17:02.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:17:02.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:17:02.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:17:02.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:17:02.905 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T01:17:02.912 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-10-06T01:17:02.929 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-10-06T01:17:02.929 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.4" 2023-10-06T01:17:02.985 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T01:17:02.986 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T01:17:02.987 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-10-06T01:17:03.005 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-10-06T01:17:03.021 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-10-06T01:17:03.021 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-10-06T01:17:03.077 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T01:17:03.077 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T01:17:03.077 INFO:teuthology.orchestra.run.smithi184.stdout:ID="rhel" 2023-10-06T01:17:03.077 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="fedora" 2023-10-06T01:17:03.077 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="8.4" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el8" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T01:17:03.078 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T01:17:03.079 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T01:17:03.079 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-10-06T01:17:03.097 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T01:17:03.107 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T01:17:03.113 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T01:17:03.113 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2023-10-06T01:17:03.116 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2023-10-06T01:17:03.132 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T01:17:03.138 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T01:17:03.138 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2023-10-06T01:17:03.172 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2023-10-06T01:17:03.186 INFO:teuthology.orchestra.run.smithi098.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T01:17:03.187 INFO:teuthology.orchestra.run.smithi184.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T01:17:03.188 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T01:17:03.243 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T01:17:03.256 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T01:17:03.257 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-06T01:17:03.257 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-06T01:17:03.257 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T01:17:03.257 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T01:17:03.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:17:03.272 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-10-06T01:17:03.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:17:03.273 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2023-10-06T01:17:03.327 INFO:teuthology.orchestra.run.smithi184.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T01:17:03.327 INFO:teuthology.task.kernel:Running kernel on smithi184: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T01:17:03.328 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2023-10-06T01:17:03.330 INFO:teuthology.orchestra.run.smithi098.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T01:17:03.330 INFO:teuthology.task.kernel:Running kernel on smithi098: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T01:17:03.330 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum install -y kernel 2023-10-06T01:17:04.765 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:04.765 INFO:teuthology.orchestra.run.smithi098.stdout:Unable to read consumer identity 2023-10-06T01:17:04.785 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:04.785 INFO:teuthology.orchestra.run.smithi184.stdout:Unable to read consumer identity 2023-10-06T01:17:06.166 INFO:teuthology.orchestra.run.smithi184.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-06T01:17:06.217 INFO:teuthology.orchestra.run.smithi098.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-06T01:17:07.066 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-10-06T01:17:08.781 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 16 MB 00:02 2023-10-06T01:17:10.360 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 4.8 MB/s | 24 kB 00:00 2023-10-06T01:17:11.089 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T01:17:11.090 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T01:17:11.090 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T01:17:11.249 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-06T01:17:11.250 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2023-10-06T01:17:11.250 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-06T01:17:11.359 DEBUG:teuthology.orchestra.run.smithi184:> echo no | sudo yum reinstall kernel || true 2023-10-06T01:17:11.702 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:11.702 INFO:teuthology.orchestra.run.smithi184.stdout:Unable to read consumer identity 2023-10-06T01:17:11.992 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 01:17:10 AM UTC. 2023-10-06T01:17:12.064 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-06T01:17:12.164 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T01:17:12.164 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:12.164 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:12.186 INFO:teuthology.orchestra.run.smithi184.stderr:Error: No packages marked for reinstall. 2023-10-06T01:17:12.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum reinstall -y kernel || true 2023-10-06T01:17:12.578 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:12.579 INFO:teuthology.orchestra.run.smithi184.stdout:Unable to read consumer identity 2023-10-06T01:17:12.797 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T01:17:12.797 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T01:17:12.798 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T01:17:12.867 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 01:17:10 AM UTC. 2023-10-06T01:17:12.959 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-10-06T01:17:12.960 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2023-10-06T01:17:12.960 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-10-06T01:17:13.008 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T01:17:13.009 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:13.009 INFO:teuthology.orchestra.run.smithi184.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:13.025 INFO:teuthology.orchestra.run.smithi184.stderr:Error: No packages marked for reinstall. 2023-10-06T01:17:13.075 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T01:17:13.088 DEBUG:teuthology.orchestra.run.smithi098:> echo no | sudo yum reinstall kernel || true 2023-10-06T01:17:13.200 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.200 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.200 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.200 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T01:17:13.200 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T01:17:13.200 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-06T01:17:13.201 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T01:17:13.201 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T01:17:13.326 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.326 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.326 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2023-10-06T01:17:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2023-10-06T01:17:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2023-10-06T01:17:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.02 2023-10-06T01:17:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 99.el8_4.1 2023-10-06T01:17:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Group : System Environment/Base 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 9264631 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Relocations : (not relocatable) 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : Red Hat, Inc. 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : Red Hat, Inc. 2023-10-06T01:17:13.354 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:17:13.355 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T01:17:13.356 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T01:17:13.356 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:13.356 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T01:17:13.428 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:13.428 INFO:teuthology.orchestra.run.smithi098.stdout:Unable to read consumer identity 2023-10-06T01:17:13.717 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 01:17:12 AM UTC. 2023-10-06T01:17:13.844 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T01:17:13.844 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:13.875 INFO:teuthology.orchestra.run.smithi098.stderr:Error: No packages marked for reinstall. 2023-10-06T01:17:13.924 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum reinstall -y kernel || true 2023-10-06T01:17:14.264 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-10-06T01:17:14.264 INFO:teuthology.orchestra.run.smithi098.stdout:Unable to read consumer identity 2023-10-06T01:17:14.397 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2023-10-06T01:17:14.550 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 01:17:12 AM UTC. 2023-10-06T01:17:14.693 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T01:17:14.694 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:14.694 INFO:teuthology.orchestra.run.smithi098.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T01:17:14.709 INFO:teuthology.orchestra.run.smithi098.stderr:Error: No packages marked for reinstall. 2023-10-06T01:17:14.758 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T01:17:14.884 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:14.884 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:14.885 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:14.885 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T01:17:14.885 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T01:17:14.885 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-06T01:17:14.885 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T01:17:14.885 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T01:17:15.010 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:15.010 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:15.010 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.02 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 99.el8_4.1 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T01:17:15.037 INFO:teuthology.orchestra.run.smithi098.stdout:Group : System Environment/Base 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 9264631 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Relocations : (not relocatable) 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : Red Hat, Inc. 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : Red Hat, Inc. 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2023-10-06T01:17:15.038 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:17:15.039 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T01:17:15.040 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T01:17:15.040 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:15.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T01:17:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2023-10-06T01:17:16.759 INFO:teuthology.orchestra.run.smithi184.stderr:done 2023-10-06T01:17:16.761 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-06T01:17:16.776 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.rLXjyWfjCJ 2023-10-06T01:17:16.776 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rLXjyWfjCJ 2023-10-06T01:17:16.845 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.rLXjyWfjCJ 2023-10-06T01:17:17.003 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.rLXjyWfjCJ is 6KB 2023-10-06T01:17:17.051 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.rLXjyWfjCJ 2023-10-06T01:17:17.065 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T01:17:17.127 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T01:17:17.127 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T01:17:17.127 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T01:17:17.127 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T01:17:17.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:17.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2023-10-06T01:17:18.646 INFO:teuthology.orchestra.run.smithi098.stderr:done 2023-10-06T01:17:18.649 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2023-10-06T01:17:18.662 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.34UJF5gGdP 2023-10-06T01:17:18.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.34UJF5gGdP 2023-10-06T01:17:18.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.34UJF5gGdP 2023-10-06T01:17:18.875 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.34UJF5gGdP is 6KB 2023-10-06T01:17:18.923 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.34UJF5gGdP 2023-10-06T01:17:18.937 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T01:17:19.002 INFO:teuthology.orchestra.run.smithi098.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T01:17:19.003 INFO:teuthology.orchestra.run.smithi098.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T01:17:19.003 INFO:teuthology.orchestra.run.smithi098.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T01:17:19.003 INFO:teuthology.orchestra.run.smithi098.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T01:17:19.004 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T01:17:19.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2023-10-06T01:17:47.305 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T01:17:47.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:17:47.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:17:49.157 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T01:17:49.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:17:49.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:18:05.690 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-10-06T01:18:07.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-06T01:18:14.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:18:14.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:18:16.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:18:16.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:18:17.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-10-06T01:18:19.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-06T01:18:29.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:18:29.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:18:31.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:18:31.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:18:49.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-06T01:19:02.442 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-10-06T01:19:03.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:19:03.510 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T01:19:03.510 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-10-06T01:19:03.523 INFO:teuthology.orchestra.run.smithi184.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T01:19:03.523 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:19:03.523 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T01:19:03.523 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T01:19:04.525 DEBUG:teuthology.parallel:result is None 2023-10-06T01:19:04.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:19:04.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:19:05.726 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-10-06T01:19:06.847 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:19:06.847 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T01:19:06.847 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2023-10-06T01:19:06.963 INFO:teuthology.orchestra.run.smithi098.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T01:19:06.964 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T01:19:06.964 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T01:19:06.964 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T01:19:07.965 DEBUG:teuthology.parallel:result is None 2023-10-06T01:19:07.965 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T01:19:07.972 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T01:19:07.973 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T01:19:07.975 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T01:19:07.993 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T01:19:07.999 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T01:19:08.007 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T01:19:08.008 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T01:19:08.031 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T01:19:08.062 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T01:19:08.069 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T01:19:08.069 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T01:19:08.090 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T01:19:08.172 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T01:19:08.190 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T01:19:08.429 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T01:19:08.447 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T01:19:08.449 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T01:19:08.456 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T01:19:08.456 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T01:19:08.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T01:19:08.487 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T01:19:08.496 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T01:19:08.496 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T01:19:08.525 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T01:19:08.543 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T01:19:08.619 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T01:19:08.739 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T01:19:08.780 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T01:19:08.818 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:19:08.818 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T01:19:08.882 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T01:19:08.916 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T01:19:08.994 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T01:19:09.036 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T01:19:09.076 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:19:09.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T01:19:09.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2023-10-06T01:19:09.144 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2023-10-06T01:19:09.195 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T01:19:09.235 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T01:19:09.432 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T01:19:09.440 INFO:teuthology.task.internal:Starting timer... 2023-10-06T01:19:09.440 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T01:19:09.450 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T01:19:09.458 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T01:19:09.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2023-10-06T01:19:09.503 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: [ OK ] 2023-10-06T01:19:09.504 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2023-10-06T01:19:09.580 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: [ OK ] 2023-10-06T01:19:09.581 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T01:19:09.582 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2023-10-06T01:19:09.611 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2023-10-06T01:19:09.611 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2023-10-06T01:19:09.629 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2023-10-06T01:19:09.629 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2023-10-06T01:19:09.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T01:19:09.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:19:09.676 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T01:19:09.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:19:09.703 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T01:19:09.703 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2023-10-06T01:19:09.749 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2023-10-06T01:19:09.774 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T01:19:09.783 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T01:19:09.789 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T01:19:09.896 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T01:19:09.910 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T01:19:09.911 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi098.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T01:36:33.642 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2023-10-06T01:36:33.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:36:33.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:36:33.717 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-10-06T01:36:33.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-06T01:36:33.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:36:33.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T01:36:33.850 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-10-06T01:36:33.914 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-06T01:36:33.914 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T01:36:33.929 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T01:36:33.929 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T01:36:33.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T01:36:33.933 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T01:36:33.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T01:36:33.962 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T01:36:33.974 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T01:36:33.983 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T01:36:33.997 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T01:36:34.008 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2023-10-06T01:36:34.018 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2023-10-06T01:36:34.032 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T01:36:34.039 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2023-10-06T01:36:34.045 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T01:36:34.049 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2023-10-06T01:36:34.064 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T01:36:34.077 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T01:36:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:210 Number of sources = 4 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.120 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.143 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-06T01:36:34.146 INFO:teuthology.orchestra.run.smithi184.stdout:210 Number of sources = 4 2023-10-06T01:36:34.146 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T01:36:34.147 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-06T01:36:34.147 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.147 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.147 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.147 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:36:34.147 INFO:teuthology.run_tasks:Running task install... 2023-10-06T01:36:34.155 DEBUG:teuthology.task.install:project ceph 2023-10-06T01:36:34.155 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T01:36:34.155 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T01:36:34.156 INFO:teuthology.task.install:Using flavor: default 2023-10-06T01:36:34.161 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T01:36:34.161 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T01:36:34.162 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T01:36:34.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T01:36:34.164 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T01:36:34.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T01:36:34.314 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T01:36:34.314 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T01:36:34.323 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T01:36:34.323 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T01:36:34.492 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T01:36:34.492 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:36:34.492 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T01:36:34.499 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T01:36:34.499 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:36:34.499 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T01:36:34.517 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T01:36:34.518 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T01:36:34.524 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T01:36:34.524 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T01:36:34.583 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T01:36:34.589 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T01:36:34.656 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T01:36:34.663 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T01:36:34.720 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2023-10-06T01:36:34.722 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2023-10-06T01:36:34.726 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2023-10-06T01:36:34.728 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2023-10-06T01:36:35.111 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-06T01:36:35.113 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-10-06T01:36:35.486 INFO:teuthology.orchestra.run.smithi098.stdout:44 files removed 2023-10-06T01:36:35.486 INFO:teuthology.orchestra.run.smithi184.stdout:44 files removed 2023-10-06T01:36:35.536 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T01:36:35.537 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T01:36:35.887 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-06T01:36:35.887 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-10-06T01:36:36.934 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 131 kB/s | 78 kB 00:00 2023-10-06T01:36:36.980 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 119 kB/s | 78 kB 00:00 2023-10-06T01:36:37.333 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 26 kB/s | 9.2 kB 00:00 2023-10-06T01:36:37.356 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 27 kB/s | 9.2 kB 00:00 2023-10-06T01:36:37.787 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 3.4 kB/s | 1.4 kB 00:00 2023-10-06T01:36:37.797 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 3.5 kB/s | 1.4 kB 00:00 2023-10-06T01:36:37.912 INFO:teuthology.orchestra.run.smithi184.stdout:Copr repo for python3-asyncssh owned by ceph 36 kB/s | 3.5 kB 00:00 2023-10-06T01:36:37.940 INFO:teuthology.orchestra.run.smithi098.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-10-06T01:36:41.363 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 16 MB 00:03 2023-10-06T01:36:42.002 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 3.9 MB/s | 16 MB 00:04 2023-10-06T01:36:44.705 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-06T01:36:45.100 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 122 MB/s | 38 MB 00:00 2023-10-06T01:36:45.251 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-06T01:36:45.650 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 120 MB/s | 38 MB 00:00 2023-10-06T01:36:50.630 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 117 MB/s | 33 MB 00:00 2023-10-06T01:36:51.148 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 115 MB/s | 33 MB 00:00 2023-10-06T01:36:58.562 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T01:36:58.563 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T01:36:58.573 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T01:36:58.574 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T01:36:58.575 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T01:36:58.576 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T01:36:58.578 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T01:36:58.580 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T01:36:58.581 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T01:36:59.074 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T01:36:59.075 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T01:36:59.084 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-10-06T01:36:59.085 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T01:36:59.086 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T01:36:59.087 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T01:36:59.088 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T01:36:59.090 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T01:36:59.092 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T01:36:59.093 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================================================== 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================================================== 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T01:36:59.096 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T01:36:59.097 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T01:36:59.098 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T01:36:59.099 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T01:36:59.100 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T01:36:59.103 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T01:36:59.104 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T01:36:59.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T01:36:59.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T01:36:59.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T01:36:59.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T01:36:59.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================================================== 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout:Install 118 Packages 2023-10-06T01:36:59.110 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 2 Packages 2023-10-06T01:36:59.111 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:36:59.111 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 201 M 2023-10-06T01:36:59.111 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-10-06T01:36:59.472 INFO:teuthology.orchestra.run.smithi098.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 41 kB/s | 6.4 kB 00:00 2023-10-06T01:36:59.582 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T01:36:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T01:36:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T01:36:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T01:36:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T01:36:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T01:36:59.598 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T01:36:59.599 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T01:36:59.600 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T01:36:59.601 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T01:36:59.602 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T01:36:59.603 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout:Install 118 Packages 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2023-10-06T01:36:59.604 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:36:59.605 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 201 M 2023-10-06T01:36:59.605 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-10-06T01:36:59.774 INFO:teuthology.orchestra.run.smithi098.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.8 MB/s | 876 kB 00:00 2023-10-06T01:36:59.821 INFO:teuthology.orchestra.run.smithi098.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 3.3 MB/s | 162 kB 00:00 2023-10-06T01:36:59.853 INFO:teuthology.orchestra.run.smithi184.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 40 kB/s | 6.4 kB 00:00 2023-10-06T01:37:00.352 INFO:teuthology.orchestra.run.smithi098.stdout:(4/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 4.3 MB/s | 2.3 MB 00:00 2023-10-06T01:37:00.584 INFO:teuthology.orchestra.run.smithi098.stdout:(5/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 4.1 MB/s | 5.2 MB 00:01 2023-10-06T01:37:00.642 INFO:teuthology.orchestra.run.smithi184.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.1 MB/s | 876 kB 00:00 2023-10-06T01:37:00.744 INFO:teuthology.orchestra.run.smithi098.stdout:(6/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 4.0 MB/s | 1.6 MB 00:00 2023-10-06T01:37:00.812 INFO:teuthology.orchestra.run.smithi184.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 952 kB/s | 162 kB 00:00 2023-10-06T01:37:01.039 INFO:teuthology.orchestra.run.smithi184.stdout:(4/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 3.8 MB/s | 5.2 MB 00:01 2023-10-06T01:37:01.284 INFO:teuthology.orchestra.run.smithi184.stdout:(5/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 6.4 MB/s | 1.6 MB 00:00 2023-10-06T01:37:01.469 INFO:teuthology.orchestra.run.smithi098.stdout:(7/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 11 MB/s | 23 MB 00:02 2023-10-06T01:37:01.521 INFO:teuthology.orchestra.run.smithi098.stdout:(8/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 4.2 MB/s | 3.9 MB 00:00 2023-10-06T01:37:01.609 INFO:teuthology.orchestra.run.smithi098.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 277 kB/s | 24 kB 00:00 2023-10-06T01:37:01.649 INFO:teuthology.orchestra.run.smithi184.stdout:(6/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 11 MB/s | 3.9 MB 00:00 2023-10-06T01:37:01.922 INFO:teuthology.orchestra.run.smithi184.stdout:(7/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-10-06T01:37:03.818 INFO:teuthology.orchestra.run.smithi098.stdout:(10/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 5.4 MB/s | 17 MB 00:03 2023-10-06T01:37:03.917 INFO:teuthology.orchestra.run.smithi098.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 2.5 MB/s | 255 kB 00:00 2023-10-06T01:37:03.997 INFO:teuthology.orchestra.run.smithi098.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 311 kB/s | 25 kB 00:00 2023-10-06T01:37:04.166 INFO:teuthology.orchestra.run.smithi098.stdout:(13/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 6.2 MB/s | 17 MB 00:02 2023-10-06T01:37:04.177 INFO:teuthology.orchestra.run.smithi098.stdout:(14/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 4.3 MB/s | 797 kB 00:00 2023-10-06T01:37:04.194 INFO:teuthology.orchestra.run.smithi098.stdout:(15/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 6.2 MB/s | 174 kB 00:00 2023-10-06T01:37:04.220 INFO:teuthology.orchestra.run.smithi184.stdout:(8/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x86 6.5 MB/s | 17 MB 00:02 2023-10-06T01:37:04.239 INFO:teuthology.orchestra.run.smithi098.stdout:(16/120): librados-devel-17.2.6-1296.g736ac5ba. 2.0 MB/s | 130 kB 00:00 2023-10-06T01:37:04.243 INFO:teuthology.orchestra.run.smithi098.stdout:(17/120): libradosstriper1-17.2.6-1296.g736ac5b 10 MB/s | 507 kB 00:00 2023-10-06T01:37:04.264 INFO:teuthology.orchestra.run.smithi184.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 555 kB/s | 24 kB 00:00 2023-10-06T01:37:04.295 INFO:teuthology.orchestra.run.smithi098.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 872 kB/s | 45 kB 00:00 2023-10-06T01:37:04.349 INFO:teuthology.orchestra.run.smithi098.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 2.1 MB/s | 115 kB 00:00 2023-10-06T01:37:04.413 INFO:teuthology.orchestra.run.smithi098.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 2.9 MB/s | 190 kB 00:00 2023-10-06T01:37:04.486 INFO:teuthology.orchestra.run.smithi098.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 5.0 MB/s | 371 kB 00:00 2023-10-06T01:37:04.568 INFO:teuthology.orchestra.run.smithi098.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.3 MB/s | 358 kB 00:00 2023-10-06T01:37:04.632 INFO:teuthology.orchestra.run.smithi098.stdout:(23/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 1.6 MB/s | 107 kB 00:00 2023-10-06T01:37:04.685 INFO:teuthology.orchestra.run.smithi098.stdout:(24/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.7 MB/s | 90 kB 00:00 2023-10-06T01:37:04.830 INFO:teuthology.orchestra.run.smithi184.stdout:(10/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 5.8 MB/s | 17 MB 00:02 2023-10-06T01:37:04.914 INFO:teuthology.orchestra.run.smithi184.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 3.0 MB/s | 255 kB 00:00 2023-10-06T01:37:05.001 INFO:teuthology.orchestra.run.smithi098.stdout:(25/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 11 MB/s | 3.5 MB 00:00 2023-10-06T01:37:05.003 INFO:teuthology.orchestra.run.smithi184.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 276 kB/s | 25 kB 00:00 2023-10-06T01:37:05.023 INFO:teuthology.orchestra.run.smithi098.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 7.8 MB/s | 173 kB 00:00 2023-10-06T01:37:05.073 INFO:teuthology.orchestra.run.smithi098.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 466 kB/s | 23 kB 00:00 2023-10-06T01:37:05.090 INFO:teuthology.orchestra.run.smithi184.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 9.0 MB/s | 797 kB 00:00 2023-10-06T01:37:05.106 INFO:teuthology.orchestra.run.smithi098.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 3.6 MB/s | 121 kB 00:00 2023-10-06T01:37:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 2.7 MB/s | 174 kB 00:00 2023-10-06T01:37:05.237 INFO:teuthology.orchestra.run.smithi184.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 1.5 MB/s | 130 kB 00:00 2023-10-06T01:37:05.287 INFO:teuthology.orchestra.run.smithi098.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 9.4 MB/s | 1.7 MB 00:00 2023-10-06T01:37:05.341 INFO:teuthology.orchestra.run.smithi184.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 4.8 MB/s | 507 kB 00:00 2023-10-06T01:37:05.794 INFO:teuthology.orchestra.run.smithi098.stdout:(30/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 3.6 MB/s | 5.6 MB 00:01 2023-10-06T01:37:05.848 INFO:teuthology.orchestra.run.smithi184.stdout:(17/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-06T01:37:05.903 INFO:teuthology.orchestra.run.smithi184.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 812 kB/s | 45 kB 00:00 2023-10-06T01:37:05.924 INFO:teuthology.orchestra.run.smithi098.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 1.8 MB/s | 238 kB 00:00 2023-10-06T01:37:05.959 INFO:teuthology.orchestra.run.smithi184.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 2.0 MB/s | 115 kB 00:00 2023-10-06T01:37:05.990 INFO:teuthology.orchestra.run.smithi098.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 717 kB/s | 47 kB 00:00 2023-10-06T01:37:06.024 INFO:teuthology.orchestra.run.smithi184.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 2.9 MB/s | 190 kB 00:00 2023-10-06T01:37:06.048 INFO:teuthology.orchestra.run.smithi098.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 250 kB/s | 14 kB 00:00 2023-10-06T01:37:06.091 INFO:teuthology.orchestra.run.smithi184.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 5.4 MB/s | 371 kB 00:00 2023-10-06T01:37:06.168 INFO:teuthology.orchestra.run.smithi184.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.6 MB/s | 358 kB 00:00 2023-10-06T01:37:06.185 INFO:teuthology.orchestra.run.smithi098.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 1.9 MB/s | 267 kB 00:00 2023-10-06T01:37:06.224 INFO:teuthology.orchestra.run.smithi184.stdout:(23/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 1.8 MB/s | 107 kB 00:00 2023-10-06T01:37:06.236 INFO:teuthology.orchestra.run.smithi098.stdout:(35/120): ceph-mgr-diskprediction-local-17.2.6- 7.8 MB/s | 7.4 MB 00:00 2023-10-06T01:37:06.248 INFO:teuthology.orchestra.run.smithi098.stdout:(36/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T01:37:06.258 INFO:teuthology.orchestra.run.smithi098.stdout:(37/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 844 kB/s | 18 kB 00:00 2023-10-06T01:37:06.275 INFO:teuthology.orchestra.run.smithi184.stdout:(24/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.7 MB/s | 90 kB 00:00 2023-10-06T01:37:06.502 INFO:teuthology.orchestra.run.smithi184.stdout:(25/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 15 MB/s | 3.5 MB 00:00 2023-10-06T01:37:06.556 INFO:teuthology.orchestra.run.smithi184.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 3.1 MB/s | 173 kB 00:00 2023-10-06T01:37:06.592 INFO:teuthology.orchestra.run.smithi098.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 176 kB/s | 59 kB 00:00 2023-10-06T01:37:06.605 INFO:teuthology.orchestra.run.smithi184.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 471 kB/s | 23 kB 00:00 2023-10-06T01:37:06.657 INFO:teuthology.orchestra.run.smithi184.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 2.3 MB/s | 121 kB 00:00 2023-10-06T01:37:06.700 INFO:teuthology.orchestra.run.smithi098.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 691 kB/s | 75 kB 00:00 2023-10-06T01:37:06.731 INFO:teuthology.orchestra.run.smithi098.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 635 kB/s | 306 kB 00:00 2023-10-06T01:37:06.792 INFO:teuthology.orchestra.run.smithi184.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 13 MB/s | 1.7 MB 00:00 2023-10-06T01:37:06.808 INFO:teuthology.orchestra.run.smithi098.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 570 kB/s | 44 kB 00:00 2023-10-06T01:37:06.910 INFO:teuthology.orchestra.run.smithi098.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-10-06T01:37:06.970 INFO:teuthology.orchestra.run.smithi098.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-10-06T01:37:06.991 INFO:teuthology.orchestra.run.smithi098.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 410 kB/s | 33 kB 00:00 2023-10-06T01:37:07.043 INFO:teuthology.orchestra.run.smithi098.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 224 kB/s | 16 kB 00:00 2023-10-06T01:37:07.088 INFO:teuthology.orchestra.run.smithi098.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-06T01:37:07.159 INFO:teuthology.orchestra.run.smithi098.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-10-06T01:37:07.185 INFO:teuthology.orchestra.run.smithi098.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 847 kB/s | 81 kB 00:00 2023-10-06T01:37:07.241 INFO:teuthology.orchestra.run.smithi098.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 135 kB/s | 11 kB 00:00 2023-10-06T01:37:07.256 INFO:teuthology.orchestra.run.smithi184.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 16 MB/s | 7.4 MB 00:00 2023-10-06T01:37:07.268 INFO:teuthology.orchestra.run.smithi098.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 213 kB/s | 18 kB 00:00 2023-10-06T01:37:07.341 INFO:teuthology.orchestra.run.smithi184.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 2.7 MB/s | 238 kB 00:00 2023-10-06T01:37:07.358 INFO:teuthology.orchestra.run.smithi098.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 539 kB/s | 49 kB 00:00 2023-10-06T01:37:07.400 INFO:teuthology.orchestra.run.smithi184.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 804 kB/s | 47 kB 00:00 2023-10-06T01:37:07.438 INFO:teuthology.orchestra.run.smithi098.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 734 kB/s | 59 kB 00:00 2023-10-06T01:37:07.461 INFO:teuthology.orchestra.run.smithi098.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 5.3 MB/s | 1.2 MB 00:00 2023-10-06T01:37:07.470 INFO:teuthology.orchestra.run.smithi184.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 208 kB/s | 14 kB 00:00 2023-10-06T01:37:07.518 INFO:teuthology.orchestra.run.smithi098.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 750 kB/s | 60 kB 00:00 2023-10-06T01:37:07.531 INFO:teuthology.orchestra.run.smithi184.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 4.3 MB/s | 267 kB 00:00 2023-10-06T01:37:07.543 INFO:teuthology.orchestra.run.smithi098.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.4 MB/s | 283 kB 00:00 2023-10-06T01:37:07.597 INFO:teuthology.orchestra.run.smithi184.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.2 MB/s | 80 kB 00:00 2023-10-06T01:37:07.598 INFO:teuthology.orchestra.run.smithi098.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 198 kB/s | 16 kB 00:00 2023-10-06T01:37:07.617 INFO:teuthology.orchestra.run.smithi098.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 454 kB/s | 34 kB 00:00 2023-10-06T01:37:07.654 INFO:teuthology.orchestra.run.smithi184.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 317 kB/s | 18 kB 00:00 2023-10-06T01:37:07.687 INFO:teuthology.orchestra.run.smithi098.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 923 kB/s | 64 kB 00:00 2023-10-06T01:37:07.698 INFO:teuthology.orchestra.run.smithi098.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-10-06T01:37:07.772 INFO:teuthology.orchestra.run.smithi098.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 220 kB/s | 19 kB 00:00 2023-10-06T01:37:07.773 INFO:teuthology.orchestra.run.smithi098.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 320 kB/s | 24 kB 00:00 2023-10-06T01:37:07.847 INFO:teuthology.orchestra.run.smithi098.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 393 kB/s | 29 kB 00:00 2023-10-06T01:37:07.848 INFO:teuthology.orchestra.run.smithi098.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 368 kB/s | 27 kB 00:00 2023-10-06T01:37:07.936 INFO:teuthology.orchestra.run.smithi098.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 2.8 MB/s | 255 kB 00:00 2023-10-06T01:37:07.967 INFO:teuthology.orchestra.run.smithi098.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-10-06T01:37:08.005 INFO:teuthology.orchestra.run.smithi098.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 890 kB/s | 61 kB 00:00 2023-10-06T01:37:08.084 INFO:teuthology.orchestra.run.smithi098.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 289 kB/s | 23 kB 00:00 2023-10-06T01:37:08.095 INFO:teuthology.orchestra.run.smithi098.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 668 kB/s | 85 kB 00:00 2023-10-06T01:37:08.108 INFO:teuthology.orchestra.run.smithi098.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 5.3 MB/s | 70 kB 00:00 2023-10-06T01:37:08.147 INFO:teuthology.orchestra.run.smithi098.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 491 kB/s | 19 kB 00:00 2023-10-06T01:37:08.173 INFO:teuthology.orchestra.run.smithi098.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.6 MB/s | 43 kB 00:00 2023-10-06T01:37:08.196 INFO:teuthology.orchestra.run.smithi098.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 6.7 MB/s | 155 kB 00:00 2023-10-06T01:37:08.207 INFO:teuthology.orchestra.run.smithi184.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 554 kB/s | 306 kB 00:00 2023-10-06T01:37:08.216 INFO:teuthology.orchestra.run.smithi098.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-06T01:37:08.234 INFO:teuthology.orchestra.run.smithi098.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.7 MB/s | 48 kB 00:00 2023-10-06T01:37:08.254 INFO:teuthology.orchestra.run.smithi098.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 11 MB/s | 229 kB 00:00 2023-10-06T01:37:08.274 INFO:teuthology.orchestra.run.smithi098.stdout:(76/120): thrift-0.13.0-2.el8.x86_64.rpm 9.1 MB/s | 1.7 MB 00:00 2023-10-06T01:37:08.276 INFO:teuthology.orchestra.run.smithi098.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 1.9 MB/s | 43 kB 00:00 2023-10-06T01:37:08.298 INFO:teuthology.orchestra.run.smithi184.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 649 kB/s | 59 kB 00:00 2023-10-06T01:37:08.303 INFO:teuthology.orchestra.run.smithi098.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 662 kB/s | 18 kB 00:00 2023-10-06T01:37:08.357 INFO:teuthology.orchestra.run.smithi098.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 28 MB/s | 2.3 MB 00:00 2023-10-06T01:37:08.362 INFO:teuthology.orchestra.run.smithi098.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-10-06T01:37:08.381 INFO:teuthology.orchestra.run.smithi098.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 3.4 MB/s | 83 kB 00:00 2023-10-06T01:37:08.384 INFO:teuthology.orchestra.run.smithi098.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 2.6 MB/s | 59 kB 00:00 2023-10-06T01:37:08.386 INFO:teuthology.orchestra.run.smithi184.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 855 kB/s | 75 kB 00:00 2023-10-06T01:37:08.408 INFO:teuthology.orchestra.run.smithi098.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 6.2 MB/s | 169 kB 00:00 2023-10-06T01:37:08.419 INFO:teuthology.orchestra.run.smithi098.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.1 MB/s | 38 kB 00:00 2023-10-06T01:37:08.429 INFO:teuthology.orchestra.run.smithi098.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.0 MB/s | 86 kB 00:00 2023-10-06T01:37:08.448 INFO:teuthology.orchestra.run.smithi098.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.0 MB/s | 58 kB 00:00 2023-10-06T01:37:08.449 INFO:teuthology.orchestra.run.smithi098.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.4 MB/s | 47 kB 00:00 2023-10-06T01:37:08.481 INFO:teuthology.orchestra.run.smithi098.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 19 MB/s | 639 kB 00:00 2023-10-06T01:37:08.484 INFO:teuthology.orchestra.run.smithi098.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 16 MB/s | 559 kB 00:00 2023-10-06T01:37:08.506 INFO:teuthology.orchestra.run.smithi098.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 8.7 MB/s | 200 kB 00:00 2023-10-06T01:37:08.511 INFO:teuthology.orchestra.run.smithi098.stdout:(91/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 2.3 MB/s | 70 kB 00:00 2023-10-06T01:37:08.540 INFO:teuthology.orchestra.run.smithi098.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 2.8 MB/s | 97 kB 00:00 2023-10-06T01:37:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-10-06T01:37:08.559 INFO:teuthology.orchestra.run.smithi098.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.2 MB/s | 44 kB 00:00 2023-10-06T01:37:08.578 INFO:teuthology.orchestra.run.smithi098.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2023-10-06T01:37:08.599 INFO:teuthology.orchestra.run.smithi098.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 7.8 MB/s | 161 kB 00:00 2023-10-06T01:37:08.623 INFO:teuthology.orchestra.run.smithi098.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 13 MB/s | 320 kB 00:00 2023-10-06T01:37:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 594 kB/s | 44 kB 00:00 2023-10-06T01:37:08.648 INFO:teuthology.orchestra.run.smithi098.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 18 MB/s | 457 kB 00:00 2023-10-06T01:37:08.717 INFO:teuthology.orchestra.run.smithi184.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.1 MB/s | 185 kB 00:00 2023-10-06T01:37:08.779 INFO:teuthology.orchestra.run.smithi098.stdout:(98/120): python3-scipy-1.0.0-20.module+el8.1.0 54 MB/s | 15 MB 00:00 2023-10-06T01:37:08.789 INFO:teuthology.orchestra.run.smithi184.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 462 kB/s | 33 kB 00:00 2023-10-06T01:37:08.798 INFO:teuthology.orchestra.run.smithi098.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.0 MB/s | 39 kB 00:00 2023-10-06T01:37:08.822 INFO:teuthology.orchestra.run.smithi098.stdout:(100/120): python3-babel-2.5.1-5.el8.noarch.rpm 28 MB/s | 4.8 MB 00:00 2023-10-06T01:37:08.829 INFO:teuthology.orchestra.run.smithi098.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 22 MB/s | 688 kB 00:00 2023-10-06T01:37:08.847 INFO:teuthology.orchestra.run.smithi098.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-10-06T01:37:08.862 INFO:teuthology.orchestra.run.smithi098.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 6.7 MB/s | 225 kB 00:00 2023-10-06T01:37:08.863 INFO:teuthology.orchestra.run.smithi184.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 223 kB/s | 16 kB 00:00 2023-10-06T01:37:08.872 INFO:teuthology.orchestra.run.smithi098.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 4.9 MB/s | 126 kB 00:00 2023-10-06T01:37:08.899 INFO:teuthology.orchestra.run.smithi098.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 6.7 MB/s | 187 kB 00:00 2023-10-06T01:37:08.925 INFO:teuthology.orchestra.run.smithi098.stdout:(106/120): python3-numpy-1.14.3-9.el8.x86_64.rp 60 MB/s | 3.7 MB 00:00 2023-10-06T01:37:08.931 INFO:teuthology.orchestra.run.smithi098.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 17 MB/s | 538 kB 00:00 2023-10-06T01:37:08.947 INFO:teuthology.orchestra.run.smithi184.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-10-06T01:37:08.951 INFO:teuthology.orchestra.run.smithi098.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 11 MB/s | 302 kB 00:00 2023-10-06T01:37:08.956 INFO:teuthology.orchestra.run.smithi098.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.1 MB/s | 157 kB 00:00 2023-10-06T01:37:08.971 INFO:teuthology.orchestra.run.smithi098.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 5.5 MB/s | 110 kB 00:00 2023-10-06T01:37:08.984 INFO:teuthology.orchestra.run.smithi098.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 13 MB/s | 353 kB 00:00 2023-10-06T01:37:08.990 INFO:teuthology.orchestra.run.smithi098.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.0 MB/s | 20 kB 00:00 2023-10-06T01:37:09.015 INFO:teuthology.orchestra.run.smithi098.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-10-06T01:37:09.017 INFO:teuthology.orchestra.run.smithi098.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 21 MB/s | 709 kB 00:00 2023-10-06T01:37:09.053 INFO:teuthology.orchestra.run.smithi184.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-10-06T01:37:09.087 INFO:teuthology.orchestra.run.smithi098.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 65 MB/s | 4.7 MB 00:00 2023-10-06T01:37:09.123 INFO:teuthology.orchestra.run.smithi098.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 43 MB/s | 4.6 MB 00:00 2023-10-06T01:37:09.127 INFO:teuthology.orchestra.run.smithi184.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-10-06T01:37:09.163 INFO:teuthology.orchestra.run.smithi098.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-10-06T01:37:09.195 INFO:teuthology.orchestra.run.smithi184.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 163 kB/s | 11 kB 00:00 2023-10-06T01:37:09.265 INFO:teuthology.orchestra.run.smithi184.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 252 kB/s | 18 kB 00:00 2023-10-06T01:37:09.442 INFO:teuthology.orchestra.run.smithi184.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-10-06T01:37:09.570 INFO:teuthology.orchestra.run.smithi098.stdout:(118/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 8.8 MB/s | 3.6 MB 00:00 2023-10-06T01:37:09.577 INFO:teuthology.orchestra.run.smithi184.stdout:(51/120): ceph-common-17.2.6-1296.g736ac5ba.el8 2.4 MB/s | 23 MB 00:09 2023-10-06T01:37:09.641 INFO:teuthology.orchestra.run.smithi184.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 244 kB/s | 49 kB 00:00 2023-10-06T01:37:09.713 INFO:teuthology.orchestra.run.smithi184.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 841 kB/s | 60 kB 00:00 2023-10-06T01:37:10.122 INFO:teuthology.orchestra.run.smithi184.stdout:(54/120): ceph-test-17.2.6-1296.g736ac5ba.el8.x 8.8 MB/s | 52 MB 00:05 2023-10-06T01:37:10.123 INFO:teuthology.orchestra.run.smithi184.stdout:(55/120): python3-more-itertools-7.2.0-3.el8.no 108 kB/s | 59 kB 00:00 2023-10-06T01:37:10.125 INFO:teuthology.orchestra.run.smithi184.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 686 kB/s | 283 kB 00:00 2023-10-06T01:37:10.203 INFO:teuthology.orchestra.run.smithi184.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-10-06T01:37:10.225 INFO:teuthology.orchestra.run.smithi184.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 331 kB/s | 34 kB 00:00 2023-10-06T01:37:10.277 INFO:teuthology.orchestra.run.smithi184.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 875 kB/s | 64 kB 00:00 2023-10-06T01:37:10.292 INFO:teuthology.orchestra.run.smithi098.stdout:(119/120): librados2-17.2.6-1296.g736ac5ba.el8. 3.0 MB/s | 3.5 MB 00:01 2023-10-06T01:37:10.318 INFO:teuthology.orchestra.run.smithi184.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 201 kB/s | 19 kB 00:00 2023-10-06T01:37:10.352 INFO:teuthology.orchestra.run.smithi184.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 319 kB/s | 24 kB 00:00 2023-10-06T01:37:10.366 INFO:teuthology.orchestra.run.smithi184.stdout:(62/120): python3-portend-2.6-1.el8.noarch.rpm 64 kB/s | 16 kB 00:00 2023-10-06T01:37:10.417 INFO:teuthology.orchestra.run.smithi184.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 297 kB/s | 29 kB 00:00 2023-10-06T01:37:10.429 INFO:teuthology.orchestra.run.smithi184.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 357 kB/s | 27 kB 00:00 2023-10-06T01:37:10.502 INFO:teuthology.orchestra.run.smithi184.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 841 kB/s | 61 kB 00:00 2023-10-06T01:37:10.578 INFO:teuthology.orchestra.run.smithi184.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-06T01:37:10.651 INFO:teuthology.orchestra.run.smithi184.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 314 kB/s | 23 kB 00:00 2023-10-06T01:37:10.652 INFO:teuthology.orchestra.run.smithi184.stdout:(68/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-10-06T01:37:10.657 INFO:teuthology.orchestra.run.smithi184.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-06T01:37:10.680 INFO:teuthology.orchestra.run.smithi184.stdout:(70/120): python3-waitress-1.4.3-1.el8.noarch.r 815 kB/s | 255 kB 00:00 2023-10-06T01:37:10.712 INFO:teuthology.orchestra.run.smithi184.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 347 kB/s | 19 kB 00:00 2023-10-06T01:37:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-06T01:37:10.731 INFO:teuthology.orchestra.run.smithi184.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 7.9 MB/s | 155 kB 00:00 2023-10-06T01:37:10.740 INFO:teuthology.orchestra.run.smithi184.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-06T01:37:10.750 INFO:teuthology.orchestra.run.smithi184.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.4 MB/s | 48 kB 00:00 2023-10-06T01:37:10.768 INFO:teuthology.orchestra.run.smithi184.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 8.1 MB/s | 229 kB 00:00 2023-10-06T01:37:10.776 INFO:teuthology.orchestra.run.smithi184.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 1.7 MB/s | 43 kB 00:00 2023-10-06T01:37:10.791 INFO:teuthology.orchestra.run.smithi184.stdout:(78/120): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-10-06T01:37:10.810 INFO:teuthology.orchestra.run.smithi184.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 531 kB/s | 18 kB 00:00 2023-10-06T01:37:10.828 INFO:teuthology.orchestra.run.smithi184.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 39 MB/s | 2.3 MB 00:00 2023-10-06T01:37:10.834 INFO:teuthology.orchestra.run.smithi184.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 3.4 MB/s | 83 kB 00:00 2023-10-06T01:37:10.836 INFO:teuthology.orchestra.run.smithi184.stdout:(82/120): pcre2-devel-10.32-2.el8.x86_64.rpm 13 MB/s | 605 kB 00:00 2023-10-06T01:37:10.848 INFO:teuthology.orchestra.run.smithi184.stdout:(83/120): elfutils-libelf-devel-0.182-3.el8.x86 2.8 MB/s | 59 kB 00:00 2023-10-06T01:37:10.874 INFO:teuthology.orchestra.run.smithi184.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 4.1 MB/s | 169 kB 00:00 2023-10-06T01:37:10.877 INFO:teuthology.orchestra.run.smithi184.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 952 kB/s | 38 kB 00:00 2023-10-06T01:37:10.889 INFO:teuthology.orchestra.run.smithi184.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.1 MB/s | 86 kB 00:00 2023-10-06T01:37:10.903 INFO:teuthology.orchestra.run.smithi184.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-10-06T01:37:10.917 INFO:teuthology.orchestra.run.smithi184.stdout:(88/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-10-06T01:37:10.935 INFO:teuthology.orchestra.run.smithi184.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 12 MB/s | 559 kB 00:00 2023-10-06T01:37:10.939 INFO:teuthology.orchestra.run.smithi184.stdout:(90/120): libgfortran-8.4.1-1.el8.x86_64.rpm 17 MB/s | 639 kB 00:00 2023-10-06T01:37:10.940 INFO:teuthology.orchestra.run.smithi184.stdout:(91/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.0 MB/s | 70 kB 00:00 2023-10-06T01:37:10.965 INFO:teuthology.orchestra.run.smithi184.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 6.6 MB/s | 200 kB 00:00 2023-10-06T01:37:10.983 INFO:teuthology.orchestra.run.smithi184.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 2.2 MB/s | 97 kB 00:00 2023-10-06T01:37:11.003 INFO:teuthology.orchestra.run.smithi184.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 1.1 MB/s | 44 kB 00:00 2023-10-06T01:37:11.015 INFO:teuthology.orchestra.run.smithi184.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 834 kB/s | 27 kB 00:00 2023-10-06T01:37:11.029 INFO:teuthology.orchestra.run.smithi184.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 6.2 MB/s | 161 kB 00:00 2023-10-06T01:37:11.045 INFO:teuthology.orchestra.run.smithi184.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 11 MB/s | 320 kB 00:00 2023-10-06T01:37:11.073 INFO:teuthology.orchestra.run.smithi184.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 10 MB/s | 457 kB 00:00 2023-10-06T01:37:11.094 INFO:teuthology.orchestra.run.smithi184.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 1.8 MB/s | 39 kB 00:00 2023-10-06T01:37:11.123 INFO:teuthology.orchestra.run.smithi184.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 23 MB/s | 688 kB 00:00 2023-10-06T01:37:11.139 INFO:teuthology.orchestra.run.smithi184.stdout:(101/120): python3-babel-2.5.1-5.el8.noarch.rpm 51 MB/s | 4.8 MB 00:00 2023-10-06T01:37:11.239 INFO:teuthology.orchestra.run.smithi184.stdout:(102/120): python3-scipy-1.0.0-20.module+el8.1. 49 MB/s | 15 MB 00:00 2023-10-06T01:37:11.242 INFO:teuthology.orchestra.run.smithi184.stdout:(103/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.1 MB/s | 259 kB 00:00 2023-10-06T01:37:11.243 INFO:teuthology.orchestra.run.smithi184.stdout:(104/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 2.1 MB/s | 225 kB 00:00 2023-10-06T01:37:11.260 INFO:teuthology.orchestra.run.smithi184.stdout:(105/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 5.9 MB/s | 126 kB 00:00 2023-10-06T01:37:11.284 INFO:teuthology.orchestra.run.smithi184.stdout:(106/120): oniguruma-6.8.2-2.el8.x86_64.rpm 4.6 MB/s | 187 kB 00:00 2023-10-06T01:37:11.318 INFO:teuthology.orchestra.run.smithi184.stdout:(107/120): socat-1.7.3.3-2.el8.x86_64.rpm 8.6 MB/s | 302 kB 00:00 2023-10-06T01:37:11.331 INFO:teuthology.orchestra.run.smithi184.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 42 MB/s | 3.7 MB 00:00 2023-10-06T01:37:11.347 INFO:teuthology.orchestra.run.smithi184.stdout:(109/120): python3-jinja2-2.10.1-2.el8_0.noarch 6.1 MB/s | 538 kB 00:00 2023-10-06T01:37:11.349 INFO:teuthology.orchestra.run.smithi184.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 4.9 MB/s | 157 kB 00:00 2023-10-06T01:37:11.370 INFO:teuthology.orchestra.run.smithi184.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 2.8 MB/s | 110 kB 00:00 2023-10-06T01:37:11.438 INFO:teuthology.orchestra.run.smithi184.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 3.8 MB/s | 353 kB 00:00 2023-10-06T01:37:11.456 INFO:teuthology.orchestra.run.smithi184.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 188 kB/s | 20 kB 00:00 2023-10-06T01:37:11.500 INFO:teuthology.orchestra.run.smithi184.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 5.3 MB/s | 709 kB 00:00 2023-10-06T01:37:11.525 INFO:teuthology.orchestra.run.smithi184.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 627 kB/s | 54 kB 00:00 2023-10-06T01:37:11.592 INFO:teuthology.orchestra.run.smithi184.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-10-06T01:37:11.684 INFO:teuthology.orchestra.run.smithi184.stdout:(117/120): openblas-0.3.12-1.el8.x86_64.rpm 25 MB/s | 4.6 MB 00:00 2023-10-06T01:37:11.778 INFO:teuthology.orchestra.run.smithi184.stdout:(118/120): openblas-threads-0.3.12-1.el8.x86_64 15 MB/s | 4.7 MB 00:00 2023-10-06T01:37:12.994 INFO:teuthology.orchestra.run.smithi184.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-10-06T01:37:13.540 INFO:teuthology.orchestra.run.smithi184.stdout:(120/120): librados2-17.2.6-1296.g736ac5ba.el8. 1.8 MB/s | 3.5 MB 00:01 2023-10-06T01:37:13.546 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-10-06T01:37:13.546 INFO:teuthology.orchestra.run.smithi184.stdout:Total 14 MB/s | 201 MB 00:13 2023-10-06T01:37:13.595 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-10-06T01:37:13.708 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-10-06T01:37:13.708 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-10-06T01:37:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-10-06T01:37:15.417 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-10-06T01:37:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-10-06T01:37:20.475 INFO:teuthology.orchestra.run.smithi098.stdout:(120/120): ceph-test-17.2.6-1296.g736ac5ba.el8. 2.7 MB/s | 52 MB 00:18 2023-10-06T01:37:20.481 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-10-06T01:37:20.482 INFO:teuthology.orchestra.run.smithi098.stdout:Total 9.4 MB/s | 201 MB 00:21 2023-10-06T01:37:20.505 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T01:37:20.535 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-10-06T01:37:20.652 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-10-06T01:37:20.652 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-10-06T01:37:20.672 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T01:37:20.822 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T01:37:21.022 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T01:37:21.404 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T01:37:21.639 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:37:22.046 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:37:22.263 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T01:37:22.382 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-10-06T01:37:22.387 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-10-06T01:37:22.394 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T01:37:22.572 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:37:22.671 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:37:22.860 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T01:37:23.163 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T01:37:23.868 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T01:37:24.197 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:37:24.313 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:37:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T01:37:24.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T01:37:24.971 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:37:25.071 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:37:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T01:37:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T01:37:25.492 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T01:37:25.735 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T01:37:25.962 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T01:37:26.122 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T01:37:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T01:37:26.382 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T01:37:26.546 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T01:37:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T01:37:27.312 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T01:37:27.597 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-10-06T01:37:27.833 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T01:37:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T01:37:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T01:37:28.051 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T01:37:28.212 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T01:37:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T01:37:28.744 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T01:37:28.824 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T01:37:29.024 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T01:37:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:37:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:37:29.708 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T01:37:29.726 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T01:37:29.849 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T01:37:29.973 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T01:37:30.076 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:37:30.135 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T01:37:30.214 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:37:30.281 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T01:37:30.448 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T01:37:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T01:37:30.788 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T01:37:30.793 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T01:37:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T01:37:31.287 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T01:37:31.485 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T01:37:31.676 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:37:31.801 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:37:32.046 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T01:37:32.211 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T01:37:32.426 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:37:32.561 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:37:32.764 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T01:37:32.848 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T01:37:33.079 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T01:37:33.326 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T01:37:33.332 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T01:37:33.583 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T01:37:33.684 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T01:37:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T01:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T01:37:34.024 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T01:37:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T01:37:34.685 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T01:37:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T01:37:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T01:37:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T01:37:35.289 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T01:37:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T01:37:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T01:37:35.921 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T01:37:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T01:37:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T01:37:36.423 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T01:37:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T01:37:36.746 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T01:37:36.801 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T01:37:36.998 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T01:37:37.189 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T01:37:37.235 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T01:37:37.369 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T01:37:37.479 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T01:37:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T01:37:37.652 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T01:37:37.732 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T01:37:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T01:37:37.917 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T01:37:38.001 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T01:37:38.174 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T01:37:38.192 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T01:37:38.348 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T01:37:38.473 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T01:37:38.513 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T01:37:38.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T01:37:38.793 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T01:37:38.852 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T01:37:39.014 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T01:37:39.075 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T01:37:39.346 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T01:37:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T01:37:39.862 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T01:37:40.154 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T01:37:40.302 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T01:37:40.481 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T01:37:40.715 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T01:37:40.960 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T01:37:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T01:37:41.181 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T01:37:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T01:37:41.479 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T01:37:41.575 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T01:37:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T01:37:41.896 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T01:37:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T01:37:42.321 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T01:37:42.466 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T01:37:42.579 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T01:37:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T01:37:42.906 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T01:37:42.921 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T01:37:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T01:37:43.203 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T01:37:43.395 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T01:37:43.439 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T01:37:43.651 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T01:37:43.678 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T01:37:43.850 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T01:37:43.860 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T01:37:44.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T01:37:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T01:37:44.401 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T01:37:44.606 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T01:37:44.823 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T01:37:45.062 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T01:37:45.279 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T01:37:45.285 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T01:37:45.482 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T01:37:45.563 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T01:37:45.584 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T01:37:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T01:37:45.810 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T01:37:45.925 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T01:37:46.147 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T01:37:46.165 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T01:37:46.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T01:37:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T01:37:46.510 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T01:37:46.685 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T01:37:46.731 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T01:37:46.861 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T01:37:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T01:37:47.034 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T01:37:47.143 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T01:37:47.227 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T01:37:47.360 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T01:37:47.554 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T01:37:47.844 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T01:37:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T01:37:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T01:37:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T01:37:48.811 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T01:37:49.105 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T01:37:49.382 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T01:37:49.631 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T01:37:49.832 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T01:37:49.987 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T01:37:50.223 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T01:37:50.474 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T01:37:50.722 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T01:37:50.952 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T01:37:51.114 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T01:37:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T01:37:51.628 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T01:37:51.880 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T01:37:52.283 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T01:37:52.493 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:37:52.564 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T01:37:52.742 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:37:52.807 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T01:37:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T01:37:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:37:53.642 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T01:37:53.719 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T01:37:53.841 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T01:37:54.255 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T01:37:54.613 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T01:37:54.883 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T01:37:55.242 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T01:37:55.532 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T01:37:55.879 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T01:37:56.102 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T01:37:56.323 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T01:37:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T01:38:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:38:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:38:02.542 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T01:38:03.336 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T01:38:03.471 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T01:38:03.588 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /run 2023-10-06T01:38:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2023-10-06T01:38:11.469 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2023-10-06T01:38:11.469 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2023-10-06T01:38:11.469 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2023-10-06T01:38:11.469 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-06T01:38:11.469 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:11.839 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T01:38:13.323 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T01:38:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T01:38:13.711 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T01:38:13.888 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T01:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T01:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T01:38:14.447 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T01:38:17.318 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T01:38:17.465 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T01:38:17.652 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T01:38:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T01:38:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T01:38:18.336 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T01:38:18.545 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T01:38:19.516 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T01:38:19.729 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T01:38:19.828 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T01:38:20.201 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T01:38:20.370 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T01:38:20.408 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T01:38:20.595 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T01:38:20.607 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /run 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2023-10-06T01:38:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2023-10-06T01:38:21.460 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2023-10-06T01:38:21.460 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-10-06T01:38:21.460 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:21.862 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T01:38:22.211 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T01:38:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T01:38:22.548 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T01:38:22.712 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T01:38:22.755 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T01:38:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T01:38:23.252 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T01:38:23.365 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T01:38:23.420 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T01:38:23.525 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T01:38:23.764 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T01:38:23.953 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T01:38:24.364 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T01:38:24.544 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T01:38:24.561 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T01:38:27.557 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T01:38:27.719 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T01:38:27.942 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T01:38:28.164 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T01:38:28.348 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T01:38:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T01:38:28.640 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T01:38:28.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T01:38:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T01:38:29.716 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T01:38:29.858 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T01:38:29.901 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 110/122 2023-10-06T01:38:30.027 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T01:38:30.119 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T01:38:30.124 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T01:38:30.268 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T01:38:30.280 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T01:38:30.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T01:38:30.495 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T01:38:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T01:38:30.715 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T01:38:30.752 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T01:38:30.760 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T01:38:30.934 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T01:38:30.945 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T01:38:30.950 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T01:38:31.190 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T01:38:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T01:38:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T01:38:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T01:38:31.635 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T01:38:31.762 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T01:38:31.762 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T01:38:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T01:38:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:32.535 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T01:38:32.790 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:38:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T01:38:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T01:38:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T01:38:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T01:38:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T01:38:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T01:38:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T01:38:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T01:38:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T01:38:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T01:38:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T01:38:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T01:38:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T01:38:32.887 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T01:38:33.077 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T01:38:33.120 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T01:38:33.301 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T01:38:33.648 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T01:38:33.860 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T01:38:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:35.001 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:43 ago on Fri 06 Oct 2023 01:36:51 AM UTC. 2023-10-06T01:38:37.161 INFO:teuthology.orchestra.run.smithi184.stdout:Installed products updated. 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T01:38:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.5-12.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T01:38:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T01:38:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T01:38:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T01:38:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T01:38:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T01:38:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T01:38:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T01:38:37.321 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T01:38:37.321 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:37.321 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-06T01:38:37.533 DEBUG:teuthology.parallel:result is None 2023-10-06T01:38:38.780 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T01:38:39.744 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T01:38:40.165 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T01:38:40.377 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 110/122 2023-10-06T01:38:40.660 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T01:38:40.875 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T01:38:40.886 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T01:38:41.046 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T01:38:41.221 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T01:38:41.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T01:38:41.624 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T01:38:41.865 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T01:38:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T01:38:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T01:38:42.342 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T01:38:42.350 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T01:38:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T01:38:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T01:38:42.769 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T01:38:42.770 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T01:38:43.688 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T01:38:43.689 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T01:38:43.690 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T01:38:43.698 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T01:38:43.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T01:38:43.700 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T01:38:43.701 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T01:38:43.702 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T01:38:43.703 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T01:38:43.704 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T01:38:43.705 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T01:38:46.122 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T01:38:46.122 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:56 ago on Fri 06 Oct 2023 01:36:50 AM UTC. 2023-10-06T01:38:48.137 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T01:38:48.309 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.310 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.311 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.5-12.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T01:38:48.312 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.313 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T01:38:48.314 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T01:38:48.315 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T01:38:48.316 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T01:38:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T01:38:48.318 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.319 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T01:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T01:38:48.321 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T01:38:48.322 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T01:38:48.322 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T01:38:48.322 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T01:38:48.322 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:48.322 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-10-06T01:38:48.529 DEBUG:teuthology.parallel:result is None 2023-10-06T01:38:48.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T01:38:48.792 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T01:38:48.814 INFO:teuthology.orchestra.run.smithi098.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T01:38:48.815 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T01:38:48.815 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T01:38:48.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T01:38:48.950 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T01:38:48.972 INFO:teuthology.orchestra.run.smithi184.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T01:38:48.972 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T01:38:48.973 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T01:38:48.973 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T01:38:48.973 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:48.973 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T01:38:49.016 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:49.016 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T01:38:49.054 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T01:38:49.054 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:49.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T01:38:49.082 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T01:38:49.147 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:49.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T01:38:49.173 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T01:38:49.240 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T01:38:49.240 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:49.240 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T01:38:49.265 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T01:38:49.330 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:49.330 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T01:38:49.353 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T01:38:49.419 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T01:38:49.419 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:49.419 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T01:38:49.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T01:38:49.507 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:49.508 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T01:38:49.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T01:38:49.596 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T01:38:49.660 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T01:38:49.660 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /var/log/ceph 2023-10-06T01:38:49.662 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2023-10-06T01:38:49.686 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T01:38:49.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T01:38:49.727 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T01:38:49.750 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T01:38:49.751 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T01:38:49.792 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T01:38:49.822 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T01:38:49.823 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T01:38:49.823 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T01:38:49.823 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:49.823 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T01:38:49.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T01:38:49.985 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T01:38:50.009 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:50.009 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T01:38:50.023 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T01:38:50.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T01:38:50.171 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T01:38:50.171 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T01:38:50.171 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412088', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi098.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPjxjWApRZqgOMmRkqAKtxkRzr0GOGEOY3Hf3J2iWiMgMQJMtCLfNCMwNhkeTMl99ekKWHz1qTVHUFzssA54Bc=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI00sWvHkKzqXh1e1/v1VTW2/GSd8VRipyNfxlH5V5o99+A1owTp6qbpism66RZ736kOpXoRoX8tHFblaGu4KLs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T01:38:50.172 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T01:38:50.174 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T01:38:50.221 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T01:38:50.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T01:38:50.287 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:50.287 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T01:38:50.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T01:38:50.308 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2023-10-06T01:38:50.364 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T01:38:50.364 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:50.364 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 221324 Links: 1 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-10-06 01:38:43.243696717 +0000 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-10-06 01:36:04.408035863 +0000 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-10-06 01:36:04.408035863 +0000 2023-10-06T01:38:50.365 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-10-06T01:38:50.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T01:38:50.427 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-10-06T01:38:50.427 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-10-06T01:38:50.427 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00023104 s, 2.2 MB/s 2023-10-06T01:38:50.428 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T01:38:50.485 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2023-10-06T01:38:50.540 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T01:38:50.540 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:50.540 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 221383 Links: 1 2023-10-06T01:38:50.540 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:50.541 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:50.541 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-10-06 01:38:43.244696703 +0000 2023-10-06T01:38:50.541 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-10-06 01:36:04.933028745 +0000 2023-10-06T01:38:50.541 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-10-06 01:36:04.933028745 +0000 2023-10-06T01:38:50.541 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-10-06T01:38:50.541 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T01:38:50.604 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-10-06T01:38:50.604 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-10-06T01:38:50.604 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000229786 s, 2.2 MB/s 2023-10-06T01:38:50.605 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T01:38:50.662 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 211956 Links: 1 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:50.717 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-10-06 01:38:43.244696703 +0000 2023-10-06T01:38:50.718 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-10-06 01:36:05.453021694 +0000 2023-10-06T01:38:50.718 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-10-06 01:36:05.453021694 +0000 2023-10-06T01:38:50.718 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-10-06T01:38:50.718 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T01:38:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-10-06T01:38:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-10-06T01:38:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00025507 s, 2.0 MB/s 2023-10-06T01:38:50.781 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T01:38:50.837 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 221482 Links: 1 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-10-06 01:38:43.244696703 +0000 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-10-06 01:36:05.994014359 +0000 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-10-06 01:36:05.994014359 +0000 2023-10-06T01:38:50.892 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-10-06T01:38:50.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T01:38:50.956 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-10-06T01:38:50.956 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-10-06T01:38:50.956 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000190107 s, 2.7 MB/s 2023-10-06T01:38:50.957 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T01:38:51.012 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:51.013 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:51.013 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T01:38:51.028 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T01:38:51.028 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2023-10-06T01:38:51.083 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T01:38:51.083 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:51.083 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 220397 Links: 1 2023-10-06T01:38:51.083 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:51.084 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:51.084 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-06 01:38:32.397569981 +0000 2023-10-06T01:38:51.084 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-06 01:26:46.940821323 +0000 2023-10-06T01:38:51.084 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-06 01:26:46.940821323 +0000 2023-10-06T01:38:51.084 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-06T01:38:51.084 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T01:38:51.145 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-06T01:38:51.145 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-06T01:38:51.146 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000249594 s, 2.1 MB/s 2023-10-06T01:38:51.147 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T01:38:51.202 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2023-10-06T01:38:51.257 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T01:38:51.257 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:51.257 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 220543 Links: 1 2023-10-06T01:38:51.257 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:51.258 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:51.258 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-06 01:38:32.397569981 +0000 2023-10-06T01:38:51.258 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-06 01:26:47.449811093 +0000 2023-10-06T01:38:51.258 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-06 01:26:47.449811093 +0000 2023-10-06T01:38:51.258 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-06T01:38:51.258 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T01:38:51.321 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-06T01:38:51.321 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-06T01:38:51.322 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00024231 s, 2.1 MB/s 2023-10-06T01:38:51.323 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T01:38:51.378 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2023-10-06T01:38:51.433 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 218661 Links: 1 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-06 01:38:32.397569981 +0000 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-06 01:26:47.924801547 +0000 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-06 01:26:47.924801547 +0000 2023-10-06T01:38:51.434 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-06T01:38:51.435 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T01:38:51.497 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-06T01:38:51.497 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-06T01:38:51.497 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00021954 s, 2.3 MB/s 2023-10-06T01:38:51.498 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T01:38:51.554 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 219538 Links: 1 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-06 01:38:32.397569981 +0000 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-06 01:26:48.409791800 +0000 2023-10-06T01:38:51.610 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-06 01:26:48.409791800 +0000 2023-10-06T01:38:51.611 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-06T01:38:51.611 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T01:38:51.673 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-06T01:38:51.673 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-06T01:38:51.673 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000176176 s, 2.9 MB/s 2023-10-06T01:38:51.674 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T01:38:51.731 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:51.731 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T01:38:51.731 INFO:tasks.ceph:Generating config... 2023-10-06T01:38:51.732 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T01:38:51.733 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T01:38:51.733 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T01:38:51.733 INFO:tasks.ceph:[client] pid file = 2023-10-06T01:38:51.733 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T01:38:51.733 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T01:38:51.733 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T01:38:51.733 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T01:38:51.733 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T01:38:51.733 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T01:38:51.733 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T01:38:51.733 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T01:38:51.734 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T01:38:51.734 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T01:38:51.734 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T01:38:51.735 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T01:38:51.735 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T01:38:51.736 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T01:38:51.778 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T01:38:51.780 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T01:38:51.860 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T01:38:51.924 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.98'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2023-10-06T01:38:51.925 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T01:38:51.926 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:51.926 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T01:38:51.978 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.98 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:52.067 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:52.067 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: generated fsid c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:setting min_mon_release = octopus 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:epoch 0 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:fsid c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:last_changed 2023-10-06T01:38:52.067709+0000 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:created 2023-10-06T01:38:52.067709+0000 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:min_mon_release 15 (octopus) 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:election_strategy: 1 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:0: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.a 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2023-10-06T01:38:52.068 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T01:38:52.070 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T01:38:52.084 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c6dd341a-820f-4e37-8897-3250639f93a1... 2023-10-06T01:38:52.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:38:52.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.169 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:38:52.170 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:38:52.171 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:38:52.172 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on skipped message = False 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2023-10-06T01:38:52.173 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2023-10-06T01:38:52.174 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2023-10-06T01:38:52.175 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 20 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 20 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 10 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:38:52.176 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:38:52.177 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2023-10-06T01:38:52.178 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:38:52.179 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: mon op complaint time = 120 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2023-10-06T01:38:52.180 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 600 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2023-10-06T01:38:52.181 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout:[mds] 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds = 20 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug frag = True 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug scatterstat = True 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: mds op complaint time = 180 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: mds verify scatter = True 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:38:52.182 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:38:52.183 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:38:52.185 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2023-10-06T01:38:52.185 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.186 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2023-10-06T01:38:52.187 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.188 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2023-10-06T01:38:52.189 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2023-10-06T01:38:52.190 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2023-10-06T01:38:52.191 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 20 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 20 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 10 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2023-10-06T01:38:52.192 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:38:52.193 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2023-10-06T01:38:52.194 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2023-10-06T01:38:52.195 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2023-10-06T01:38:52.196 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2023-10-06T01:38:52.197 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = True 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:38:52.198 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:38:52.198 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T01:38:52.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T01:38:52.277 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T01:38:52.277 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:52.277 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T01:38:52.292 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:52.293 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T01:38:52.346 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:38:52.346 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:52.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T01:38:52.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T01:38:52.423 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:38:52.424 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:52.477 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:38:52.478 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:52.478 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T01:38:52.478 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T01:38:52.514 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:38:52.514 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:52.569 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T01:38:52.569 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T01:38:52.569 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T01:38:52.617 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T01:38:52.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T01:38:52.687 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T01:38:52.689 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T01:38:52.690 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T01:38:52.740 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T01:38:52.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T01:38:52.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T01:38:52.856 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T01:38:52.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T01:38:52.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T01:38:52.933 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T01:38:52.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T01:38:52.959 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T01:38:53.047 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T01:38:53.049 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T01:38:53.073 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T01:38:53.073 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T01:38:53.111 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T01:38:53.124 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T01:38:53.201 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T01:38:53.214 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T01:38:53.251 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T01:38:53.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T01:38:53.342 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T01:38:53.355 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T01:38:53.355 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T01:38:53.355 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T01:38:53.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:53.380 INFO:tasks.ceph:role: osd.0 2023-10-06T01:38:53.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:38:53.380 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T01:38:53.452 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:38:53.452 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:38:53.452 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:38:53.452 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:38:53.453 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:38:53.878 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2023-10-06T01:38:53.880 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2023-10-06T01:38:53.880 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T01:38:54.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T01:38:54.081 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T01:38:54.147 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:54.147 INFO:tasks.ceph:role: osd.1 2023-10-06T01:38:54.147 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:38:54.147 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T01:38:54.218 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:38:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:38:54.646 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2023-10-06T01:38:54.647 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2023-10-06T01:38:54.648 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T01:38:54.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T01:38:54.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T01:38:54.842 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:54.843 INFO:tasks.ceph:role: osd.2 2023-10-06T01:38:54.843 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:38:54.843 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T01:38:54.914 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:38:54.914 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:38:54.915 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:38:55.364 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2023-10-06T01:38:55.365 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2023-10-06T01:38:55.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T01:38:55.515 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T01:38:55.545 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T01:38:55.610 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T01:38:55.610 INFO:tasks.ceph:role: osd.3 2023-10-06T01:38:55.610 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:38:55.610 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:38:55.684 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:38:55.685 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:38:55.685 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:38:55.685 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:38:56.130 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2023-10-06T01:38:56.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2023-10-06T01:38:56.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T01:38:56.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T01:38:56.323 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:56.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.473+0000 7fe0c0e66540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T01:38:56.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.473+0000 7fe0c0e66540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T01:38:56.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.473+0000 7fe0c0e66540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T01:38:56.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.473+0000 7fe0c0e66540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T01:38:56.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.473+0000 7fe0c0e66540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T01:38:56.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:56.474+0000 7fe0c0e66540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T01:38:57.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:57.061+0000 7fe0c0e66540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:38:57.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:57.784+0000 7fe0c0e66540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:38:58.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:58.557+0000 7fe0c0e66540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:38:59.370 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T01:38:59.395 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.476+0000 7f023ee30540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.476+0000 7f023ee30540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.476+0000 7f023ee30540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.476+0000 7f023ee30540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.476+0000 7f023ee30540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T01:38:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:38:59.477+0000 7f023ee30540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T01:39:00.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:00.007+0000 7f023ee30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:00.786 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:00.785+0000 7f023ee30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:01.561+0000 7f023ee30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:02.370 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T01:39:02.435 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:02.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.518+0000 7f261b009540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T01:39:02.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.519+0000 7f261b009540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T01:39:02.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.519+0000 7f261b009540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T01:39:02.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.519+0000 7f261b009540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T01:39:02.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.519+0000 7f261b009540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T01:39:02.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:02.520+0000 7f261b009540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T01:39:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:03.050+0000 7f261b009540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:03.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:03.829+0000 7f261b009540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:04.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:04.599+0000 7f261b009540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:05.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T01:39:05.441 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:05.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T01:39:05.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T01:39:05.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T01:39:05.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T01:39:05.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T01:39:05.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:05.523+0000 7ff4d18f0540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T01:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:06.045+0000 7ff4d18f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:06.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:06.816+0000 7ff4d18f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:07.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:07.592+0000 7ff4d18f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:08.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T01:39:08.471 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T01:39:08.499 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T01:39:08.499 INFO:tasks.ceph:role: osd.4 2023-10-06T01:39:08.499 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:39:08.500 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T01:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:39:09.006 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-06T01:39:09.007 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-06T01:39:09.008 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T01:39:09.194 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T01:39:09.226 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T01:39:09.291 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T01:39:09.291 INFO:tasks.ceph:role: osd.5 2023-10-06T01:39:09.291 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:39:09.291 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:39:09.363 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:39:09.364 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:39:09.786 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-06T01:39:09.788 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-06T01:39:09.788 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T01:39:09.892 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T01:39:09.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T01:39:09.984 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T01:39:09.984 INFO:tasks.ceph:role: osd.6 2023-10-06T01:39:09.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:39:09.984 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:39:10.056 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:39:10.057 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:39:10.057 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:39:10.507 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-06T01:39:10.508 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-06T01:39:10.508 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T01:39:10.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T01:39:10.710 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T01:39:10.774 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T01:39:10.774 INFO:tasks.ceph:role: osd.7 2023-10-06T01:39:10.774 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:39:10.774 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T01:39:10.846 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T01:39:10.847 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T01:39:11.285 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-06T01:39:11.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-06T01:39:11.287 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T01:39:11.474 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T01:39:11.503 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.597+0000 7f90988d0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.598+0000 7f90988d0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.598+0000 7f90988d0540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.598+0000 7f90988d0540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.598+0000 7f90988d0540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T01:39:11.599 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:11.598+0000 7f90988d0540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T01:39:12.123 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:12.122+0000 7f90988d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:12.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:12.887+0000 7f90988d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:13.650 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:13.649+0000 7f90988d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:14.446 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T01:39:14.471 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:14.552 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.550+0000 7fe694d33540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T01:39:14.553 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.550+0000 7fe694d33540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T01:39:14.553 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.550+0000 7fe694d33540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T01:39:14.553 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.550+0000 7fe694d33540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T01:39:14.553 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.550+0000 7fe694d33540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T01:39:14.553 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:14.551+0000 7fe694d33540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T01:39:15.077 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:15.076+0000 7fe694d33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:15.840 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:15.839+0000 7fe694d33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:16.605 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:16.604+0000 7fe694d33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:17.395 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T01:39:17.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:17.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.542+0000 7f34f2828540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T01:39:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.542+0000 7f34f2828540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T01:39:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.542+0000 7f34f2828540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T01:39:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.542+0000 7f34f2828540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T01:39:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.542+0000 7f34f2828540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T01:39:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:17.543+0000 7f34f2828540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T01:39:18.067 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:18.066+0000 7f34f2828540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:18.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:18.830+0000 7f34f2828540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:19.594 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:19.593+0000 7f34f2828540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:20.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:39:20.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:39:20.387 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T01:39:20.411 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:20.491 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T01:39:20.491 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T01:39:20.491 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T01:39:20.491 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T01:39:20.491 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T01:39:20.492 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:20.490+0000 7fba645d1540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T01:39:21.013 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:21.012+0000 7fba645d1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:21.782 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:21.781+0000 7fba645d1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:22.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:22.544+0000 7fba645d1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T01:39:23.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T01:39:23.413 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T01:39:23.413 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.413 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T01:39:23.439 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T01:39:23.504 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T01:39:23.569 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.569 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T01:39:23.634 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.634 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T01:39:23.699 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.699 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T01:39:23.764 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:23.764 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T01:39:23.828 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:23.828 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T01:39:23.857 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:23.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T01:39:23.923 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:23.923 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T01:39:23.990 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:23.990 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T01:39:24.055 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:24.055 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T01:39:24.119 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:24.120 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T01:39:24.184 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:24.184 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T01:39:24.248 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:24.248 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T01:39:24.262 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:24.262 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T01:39:24.317 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:24.317 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T01:39:24.331 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:24.332 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T01:39:24.386 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T01:39:24.386 DEBUG:teuthology.orchestra.run.smithi098:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T01:39:24.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T01:39:24.429 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.y] 2023-10-06T01:39:24.429 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAsZR9ljVHOJBAAhQ5RLlxGti1y0Db8NuigUA== 2023-10-06T01:39:24.429 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2023-10-06T01:39:24.429 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAsZR9lPgAnLBAAjPpez3a4NMXMOAj1v9I0ug== 2023-10-06T01:39:24.429 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAsZR9loeoKMxAAGFFjEHx40kNFdSV46mowsQ== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.0] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAwZR9lbBlGHBAAq23Jb6irVfH1CrM8J54NGQ== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.1] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAzZR9liLBvHBAABkFDdQyx6FyBrj7O4jKsQw== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.2] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA2ZR9lCFH5HhAAPe1eTYOZnHOb1K264zFY0A== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.3] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA5ZR9lyco2HxAAnSRZPU4vlmlkn4/obvlWeA== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.x] 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAsZR9lJRf+KBAAg9krytJ71YkrNr+tMUBirg== 2023-10-06T01:39:24.430 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAsZR9lKhKpNxAAynwQ7/OfekKfI1DgEHcXAA== 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAtZR9lcDnUAhAA/LRnWVjCf3o19pvkMLfjXQ== 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.4] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA/ZR9lRZC1IxAAlrj4NHXrXoocVCesey7/TA== 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.5] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBCZR9lnO3kIBAAvyVLJH3R3hrfMUQwBr2xpA== 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.6] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBFZR9lubpmIBAADMQvrtBlCG7dq7NCJDynqw== 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.7] 2023-10-06T01:39:24.431 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBIZR9lBlpJHRAAX1mz3qw8W8JKNufZkolpkg== 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAtZR9lSDqnBhAAtiG2UleyTmLx7DKNixS1iA== 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout:[client.2] 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAtZR9lieoMDBAAi/Gm21ZFDpqR66eX6Hq1Ow== 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout:[client.1] 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAtZR9lOaUGDxAAjyXgHmQqX6NPEqANIZaUkg== 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout:[client.3] 2023-10-06T01:39:24.432 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAtZR9lIXluFBAAGvd+AqaxvZ1SvnUacw8j4w== 2023-10-06T01:39:24.448 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2023-10-06T01:39:24.448 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsZR9ljVHOJBAAhQ5RLlxGti1y0Db8NuigUA== 2023-10-06T01:39:24.448 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-06T01:39:24.448 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsZR9lPgAnLBAAjPpez3a4NMXMOAj1v9I0ug== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsZR9loeoKMxAAGFFjEHx40kNFdSV46mowsQ== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAwZR9lbBlGHBAAq23Jb6irVfH1CrM8J54NGQ== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAzZR9liLBvHBAABkFDdQyx6FyBrj7O4jKsQw== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQA2ZR9lCFH5HhAAPe1eTYOZnHOb1K264zFY0A== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQA5ZR9lyco2HxAAnSRZPU4vlmlkn4/obvlWeA== 2023-10-06T01:39:24.449 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsZR9lJRf+KBAAg9krytJ71YkrNr+tMUBirg== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsZR9lKhKpNxAAynwQ7/OfekKfI1DgEHcXAA== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtZR9lcDnUAhAA/LRnWVjCf3o19pvkMLfjXQ== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQA/ZR9lRZC1IxAAlrj4NHXrXoocVCesey7/TA== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBCZR9lnO3kIBAAvyVLJH3R3hrfMUQwBr2xpA== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBFZR9lubpmIBAADMQvrtBlCG7dq7NCJDynqw== 2023-10-06T01:39:24.450 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBIZR9lBlpJHRAAX1mz3qw8W8JKNufZkolpkg== 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtZR9lSDqnBhAAtiG2UleyTmLx7DKNixS1iA== 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtZR9lieoMDBAAi/Gm21ZFDpqR66eX6Hq1Ow== 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtZR9lOaUGDxAAjyXgHmQqX6NPEqANIZaUkg== 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2023-10-06T01:39:24.451 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtZR9lIXluFBAAGvd+AqaxvZ1SvnUacw8j4w== 2023-10-06T01:39:24.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T01:39:24.476 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T01:39:24.538 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.648 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.695 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.745 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.797 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:24.844 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T01:39:24.846 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T01:39:24.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.895 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.945 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T01:39:24.998 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.002 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.097 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.147 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.149 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T01:39:25.196 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.198 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.250 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.300 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.352 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T01:39:25.404 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T01:39:25.404 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T01:39:25.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T01:39:25.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T01:39:25.543 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T01:39:25.568 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T01:39:25.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T01:39:25.683 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T01:39:25.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T01:39:25.838 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T01:39:25.862 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:25.864 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T01:39:25.917 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T01:39:25.918 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T01:39:25.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T01:39:25.920 INFO:tasks.ceph.mon.a:Started 2023-10-06T01:39:25.920 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T01:39:25.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T01:39:25.960 INFO:tasks.ceph.mon.b:Started 2023-10-06T01:39:25.960 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T01:39:25.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T01:39:25.962 INFO:tasks.ceph.mon.c:Started 2023-10-06T01:39:25.962 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T01:39:25.963 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T01:39:25.963 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T01:39:25.965 INFO:tasks.ceph.mgr.y:Started 2023-10-06T01:39:25.965 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T01:39:25.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T01:39:25.967 INFO:tasks.ceph.mgr.x:Started 2023-10-06T01:39:25.967 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T01:39:25.968 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T01:39:26.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.083+0000 7f020b5af700 1 Processor -- start 2023-10-06T01:39:26.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.083+0000 7f020b5af700 1 -- start start 2023-10-06T01:39:26.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020b5af700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f0204067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020b5af700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020406b1b0 con 0x7f0204142f20 2023-10-06T01:39:26.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020b5af700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0204067f80 con 0x7f02040c7830 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020b5af700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02041432f0 con 0x7f02040675f0 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020934b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0204142f20 0x7f020406b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38090/0 (socket says 172.21.15.98:38090) 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.084+0000 7f020934b700 1 -- 172.21.15.98:0/2122874959 learned_addr learned my addr 172.21.15.98:0/2122874959 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.085+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f0204067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.085+0000 7f0208b4a700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 msgr2=0x7f0204067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.085+0000 7f0208b4a700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 msgr2=0x7f0204067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.085+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f0204067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T01:39:26.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.085+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f0204067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T01:39:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.267+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174748132 0 0) 0x7f020406b1b0 con 0x7f0204142f20 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.267+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f0003570 con 0x7f0204142f20 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073592006 0 0) 0x7f01f0003570 con 0x7f0204142f20 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 msgr2=0x7f0204067a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f0204067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 >> v1:172.21.15.184:6789/0 conn(0x7f02040c7830 legacy=0x7f02040cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040cba10 con 0x7f0204142f20 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f01f8001470 con 0x7f0204142f20 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 >> v1:172.21.15.98:6789/0 conn(0x7f0204142f20 legacy=0x7f020406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f0003a90 0x7f01f0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0004810 0x7f01f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f01f0005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02041432f0 con 0x7f01f0003a90 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000bc20 con 0x7f01f0004810 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000bde0 con 0x7f02040675f0 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f0003a90 0x7f01f0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f0209b4c700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f01f0005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0004810 0x7f01f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 msgr2=0x7f01f0005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02040675f0 0x7f01f0005290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0004810 msgr2=0x7f01f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0004810 0x7f01f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 -- 172.21.15.98:0/2122874959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0003570 con 0x7f01f0003a90 2023-10-06T01:39:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0004810 0x7f01f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f020934b700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f0003a90 0x7f01f0003ea0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c310 tx=0x7f02040cbba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fd1fbf0c6b39a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800cc40 con 0x7f01f0003a90 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f800cdc0 con 0x7f01f0003a90 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.268+0000 7f01f7fff700 1 -- 172.21.15.98:0/2122874959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8005b70 con 0x7f01f0003a90 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.269+0000 7f020b5af700 1 -- 172.21.15.98:0/2122874959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f0003a90 msgr2=0x7f01f0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.269+0000 7f020b5af700 1 --2- 172.21.15.98:0/2122874959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01f0003a90 0x7f01f0003ea0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c310 tx=0x7f02040cbba0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.269+0000 7f0209b4c700 1 -- 172.21.15.98:0/2122874959 reap_dead start 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.269+0000 7f020b5af700 1 -- 172.21.15.98:0/2122874959 shutdown_connections 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.269+0000 7f020b5af700 1 -- 172.21.15.98:0/2122874959 >> 172.21.15.98:0/2122874959 conn(0x7f02040cd1f0 msgr2=0x7f02040cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.270+0000 7f020b5af700 1 -- 172.21.15.98:0/2122874959 shutdown_connections 2023-10-06T01:39:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.270+0000 7f020b5af700 1 -- 172.21.15.98:0/2122874959 wait complete. 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.271+0000 7f020b5af700 1 Processor -- start 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.271+0000 7f020b5af700 1 -- start start 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0003a90 0x7f020413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f020413d150 0x7f020413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02040c85e0 con 0x7f02040c7c30 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020413e760 con 0x7f01f0003a90 2023-10-06T01:39:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020b5af700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020413dec0 con 0x7f020413d150 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44286/0 (socket says 172.21.15.98:44286) 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 -- 172.21.15.98:0/2283428838 learned_addr learned my addr 172.21.15.98:0/2283428838 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f020934b700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0003a90 0x7f020413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 -- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f020413d150 msgr2=0x7f020413d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f020413d150 0x7f020413d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 -- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0003a90 msgr2=0x7f020413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0003a90 0x7f020413eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 -- 172.21.15.98:0/2283428838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020413e1a0 con 0x7f02040c7c30 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.273+0000 7f0208b4a700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f020000e8d0 tx=0x7f0200008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200013410 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200013570 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020413e480 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200013910 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020413db10 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0200013cc0 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.275+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f020001cd90 con 0x7f02040c7c30 2023-10-06T01:39:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.276+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0204140260 con 0x7f02040c7c30 2023-10-06T01:39:26.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.279+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0204140260 con 0x7f02040c7c30 2023-10-06T01:39:26.357 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.356+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0204064ad0 con 0x7f02040c7c30 2023-10-06T01:39:26.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.363+0000 7f01f57fa700 1 -- 172.21.15.98:0/2283428838 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0200021060 con 0x7f02040c7c30 2023-10-06T01:39:26.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 msgr2=0x7f02040c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f020000e8d0 tx=0x7f0200008040 comp rx=0 tx=0).stop 2023-10-06T01:39:26.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 shutdown_connections 2023-10-06T01:39:26.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f020413d150 0x7f020413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f0003a90 0x7f020413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 --2- 172.21.15.98:0/2283428838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02040c7c30 0x7f02040c80a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 >> 172.21.15.98:0/2283428838 conn(0x7f02040cd1f0 msgr2=0x7f0204158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 shutdown_connections 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:26.364+0000 7f020b5af700 1 -- 172.21.15.98:0/2283428838 wait complete. 2023-10-06T01:39:26.366 INFO:teuthology.orchestra.run.smithi098.stderr:adjusted tunables profile to default 2023-10-06T01:39:26.376 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T01:39:26.377 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:26.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T01:39:26.401 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:26.401 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T01:39:26.464 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:26.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T01:39:26.528 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-10-06T01:39:26.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T01:39:26.592 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:26.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T01:39:26.617 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:26.617 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T01:39:26.682 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:26.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T01:39:26.747 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-06T01:39:26.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T01:39:26.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new d4844d71-6e27-46c9-9909-2d60d00e922a 0 2023-10-06T01:39:26.967 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.966+0000 7f1b67b8e700 1 Processor -- start 2023-10-06T01:39:26.968 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 -- start start 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b600b0bc0 con 0x7f1b600c7ea0 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b600c9e90 con 0x7f1b6006ad60 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.967+0000 7f1b67b8e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6006b720 con 0x7f1b60156690 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6612b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6612b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38722/0 (socket says 172.21.15.184:38722) 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:56724/0 (socket says 172.21.15.184:56724) 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 -- 172.21.15.184:0/3300933161 learned_addr learned my addr 172.21.15.184:0/3300933161 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 -- 172.21.15.184:0/3300933161 >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 msgr2=0x7f1b6006b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 --2- 172.21.15.184:0/3300933161 >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 -- 172.21.15.184:0/3300933161 >> v1:172.21.15.98:6789/0 conn(0x7f1b600c7ea0 legacy=0x7f1b600b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 -- 172.21.15.184:0/3300933161 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600c8310 con 0x7f1b60156690 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6612b700 1 --2- 172.21.15.184:0/3300933161 >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b6592a700 1 --2- 172.21.15.184:0/3300933161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b500099b0 tx=0x7f1b50009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72895e5566fcc30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b64928700 1 -- 172.21.15.184:0/3300933161 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50009ea0 con 0x7f1b60156690 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b64928700 1 -- 172.21.15.184:0/3300933161 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5000b860 con 0x7f1b60156690 2023-10-06T01:39:26.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.968+0000 7f1b64928700 1 -- 172.21.15.184:0/3300933161 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000bc00 con 0x7f1b60156690 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 -- 172.21.15.184:0/3300933161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 msgr2=0x7f1b600c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/3300933161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b500099b0 tx=0x7f1b50009250 comp rx=0 tx=0).stop 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 -- 172.21.15.184:0/3300933161 shutdown_connections 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/3300933161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b60156690 0x7f1b600c7960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/3300933161 >> v2:172.21.15.184:3300/0 conn(0x7f1b6006ad60 0x7f1b6006b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 -- 172.21.15.184:0/3300933161 >> 172.21.15.184:0/3300933161 conn(0x7f1b600cd430 msgr2=0x7f1b600c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 -- 172.21.15.184:0/3300933161 shutdown_connections 2023-10-06T01:39:26.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.969+0000 7f1b67b8e700 1 -- 172.21.15.184:0/3300933161 wait complete. 2023-10-06T01:39:26.974 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.970+0000 7f1b67b8e700 1 Processor -- start 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.970+0000 7f1b67b8e700 1 -- start start 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b6006ad60 0x7f1b6016aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 0x7f1b60144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6016af60 con 0x7f1b6006ad60 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6016d5b0 con 0x7f1b60156690 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b67b8e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6016d800 con 0x7f1b600c7ea0 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b65129700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 0x7f1b60144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.975 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38728/0 (socket says 172.21.15.184:38728) 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 -- 172.21.15.184:0/2312154090 learned_addr learned my addr 172.21.15.184:0/2312154090 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6592a700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b6006ad60 0x7f1b6016aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 -- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 msgr2=0x7f1b60144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 0x7f1b60144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 -- 172.21.15.184:0/2312154090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b6006ad60 msgr2=0x7f1b6016aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b6006ad60 0x7f1b6016aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 -- 172.21.15.184:0/2312154090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009a20 con 0x7f1b60156690 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b65129700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 0x7f1b60144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b6612b700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c0f0 tx=0x7f1b5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800cc30 con 0x7f1b60156690 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.971+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5800f040 con 0x7f1b60156690 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.972+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500094f0 con 0x7f1b60156690 2023-10-06T01:39:26.976 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.972+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58014480 con 0x7f1b60156690 2023-10-06T01:39:26.977 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.972+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1b58003020 con 0x7f1b60156690 2023-10-06T01:39:26.977 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.972+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6016ddc0 con 0x7f1b60156690 2023-10-06T01:39:26.977 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.972+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1b5801d9e0 con 0x7f1b60156690 2023-10-06T01:39:26.977 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.973+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b60064ad0 con 0x7f1b60156690 2023-10-06T01:39:26.977 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:26.976+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1b58018030 con 0x7f1b60156690 2023-10-06T01:39:27.060 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.059+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4844d71-6e27-46c9-9909-2d60d00e922a", "id": 0} v 0) v1 -- 0x7f1b600c85c0 con 0x7f1b60156690 2023-10-06T01:39:27.063 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.062+0000 7f1b55ffb700 1 -- 172.21.15.184:0/2312154090 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4844d71-6e27-46c9-9909-2d60d00e922a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1b58014d20 con 0x7f1b60156690 2023-10-06T01:39:27.063 INFO:teuthology.orchestra.run.smithi184.stdout:0 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 msgr2=0x7f1b6014a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c0f0 tx=0x7f1b5800b040 comp rx=0 tx=0).stop 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 shutdown_connections 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b60156690 0x7f1b6014a440 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b6006ad60 0x7f1b6016aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 --2- 172.21.15.184:0/2312154090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b600c7ea0 0x7f1b60144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.063+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 >> 172.21.15.184:0/2312154090 conn(0x7f1b600cd430 msgr2=0x7f1b600c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.064+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 shutdown_connections 2023-10-06T01:39:27.065 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.064+0000 7f1b67b8e700 1 -- 172.21.15.184:0/2312154090 wait complete. 2023-10-06T01:39:27.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 77069145-8b7e-4c1f-a831-9497591970cc 1 2023-10-06T01:39:27.192 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 Processor -- start 2023-10-06T01:39:27.192 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 -- start start 2023-10-06T01:39:27.194 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306415b980 0x7f306406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.194 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3064158420 0x7f3064158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.194 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3064158820 con 0x7f306406be20 2023-10-06T01:39:27.194 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306406c2c0 con 0x7f3064158420 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f3069ce4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30640afbf0 con 0x7f306415b980 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.191+0000 7f30637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306415b980 0x7f306406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306415b980 0x7f306406b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38730/0 (socket says 172.21.15.184:38730) 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 -- 172.21.15.184:0/1772625445 learned_addr learned my addr 172.21.15.184:0/1772625445 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f3063fff700 1 --2- 172.21.15.184:0/1772625445 >> v2:172.21.15.98:3300/0 conn(0x7f3064158420 0x7f3064158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 -- 172.21.15.184:0/1772625445 >> v2:172.21.15.98:3300/0 conn(0x7f3064158420 msgr2=0x7f3064158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 --2- 172.21.15.184:0/1772625445 >> v2:172.21.15.98:3300/0 conn(0x7f3064158420 0x7f3064158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 -- 172.21.15.184:0/1772625445 >> v1:172.21.15.98:6789/0 conn(0x7f306406be20 legacy=0x7f3064157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 -- 172.21.15.184:0/1772625445 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30640afed0 con 0x7f306415b980 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30637fe700 1 --2- 172.21.15.184:0/1772625445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306415b980 0x7f306406b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f306406f1f0 tx=0x7f3054009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cb6e9a3269de4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30627fc700 1 -- 172.21.15.184:0/1772625445 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054009e70 con 0x7f306415b980 2023-10-06T01:39:27.195 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30627fc700 1 -- 172.21.15.184:0/1772625445 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400b860 con 0x7f306415b980 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.192+0000 7f30627fc700 1 -- 172.21.15.184:0/1772625445 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305400bc20 con 0x7f306415b980 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 -- 172.21.15.184:0/1772625445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306415b980 msgr2=0x7f306406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 --2- 172.21.15.184:0/1772625445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306415b980 0x7f306406b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f306406f1f0 tx=0x7f3054009250 comp rx=0 tx=0).stop 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 -- 172.21.15.184:0/1772625445 shutdown_connections 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 --2- 172.21.15.184:0/1772625445 >> v2:172.21.15.98:3300/0 conn(0x7f3064158420 0x7f3064158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 --2- 172.21.15.184:0/1772625445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306415b980 0x7f306406b8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 -- 172.21.15.184:0/1772625445 >> 172.21.15.184:0/1772625445 conn(0x7f30640ce330 msgr2=0x7f30640ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 -- 172.21.15.184:0/1772625445 shutdown_connections 2023-10-06T01:39:27.196 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.193+0000 7f3069ce4700 1 -- 172.21.15.184:0/1772625445 wait complete. 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.194+0000 7f3069ce4700 1 Processor -- start 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.194+0000 7f3069ce4700 1 -- start start 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f306406be20 0x7f3064168750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306415b980 0x7f306414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641682c0 con 0x7f306406be20 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064167cb0 con 0x7f3064158420 2023-10-06T01:39:27.199 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3069ce4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30640b3650 con 0x7f306415b980 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3063fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306415b980 0x7f306414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:38744/0 (socket says 172.21.15.184:38744) 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 -- 172.21.15.184:0/4175322250 learned_addr learned my addr 172.21.15.184:0/4175322250 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f30637fe700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f306406be20 0x7f3064168750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 -- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306415b980 msgr2=0x7f306414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306415b980 0x7f306414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 -- 172.21.15.184:0/4175322250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f306406be20 msgr2=0x7f3064168750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f306406be20 0x7f3064168750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 -- 172.21.15.184:0/4175322250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540094f0 con 0x7f3064158420 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f3062ffd700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f305800c590 tx=0x7f305800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.200 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.195+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058011590 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058011710 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3064167f50 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058011ad0 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30640c4af0 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f305801c6c0 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.196+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f30580127e0 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.197+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30640b0ab0 con 0x7f3064158420 2023-10-06T01:39:27.201 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.199+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3058017070 con 0x7f3064158420 2023-10-06T01:39:27.283 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.282+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "77069145-8b7e-4c1f-a831-9497591970cc", "id": 1} v 0) v1 -- 0x7f30640b08b0 con 0x7f3064158420 2023-10-06T01:39:27.286 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.285+0000 7f304b7fe700 1 -- 172.21.15.184:0/4175322250 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "77069145-8b7e-4c1f-a831-9497591970cc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f305801cb50 con 0x7f3064158420 2023-10-06T01:39:27.286 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-06T01:39:27.287 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 msgr2=0x7f30640cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f305800c590 tx=0x7f305800c8a0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 shutdown_connections 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306415b980 0x7f306414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f306406be20 0x7f3064168750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 --2- 172.21.15.184:0/4175322250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3064158420 0x7f30640cd870 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 >> 172.21.15.184:0/4175322250 conn(0x7f30640ce330 msgr2=0x7f30640c3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 shutdown_connections 2023-10-06T01:39:27.288 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.286+0000 7f3069ce4700 1 -- 172.21.15.184:0/4175322250 wait complete. 2023-10-06T01:39:27.299 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new e6c83413-b7df-4e9a-8c56-826e9ea15c07 2 2023-10-06T01:39:27.414 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.413+0000 7f564a449700 1 Processor -- start 2023-10-06T01:39:27.415 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.413+0000 7f564a449700 1 -- start start 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f564a449700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56440b2a90 0x7f56440cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f564a449700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 0x7f56440cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f564a449700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56440cbdf0 con 0x7f56440cb980 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f564a449700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440b2e60 con 0x7f56440c6610 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f564a449700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440cf5d0 con 0x7f56440b2a90 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5643fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56440b2a90 0x7f56440cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f56437fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f56440cb980 0x7f56440ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42544/0 (socket says 172.21.15.184:42544) 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f56437fe700 1 -- 172.21.15.184:0/3921994179 learned_addr learned my addr 172.21.15.184:0/3921994179 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f56489e6700 1 --2- 172.21.15.184:0/3921994179 >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 0x7f56440cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5643fff700 1 -- 172.21.15.184:0/3921994179 >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 msgr2=0x7f56440cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5643fff700 1 --2- 172.21.15.184:0/3921994179 >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 0x7f56440cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5643fff700 1 -- 172.21.15.184:0/3921994179 >> v1:172.21.15.184:6789/0 conn(0x7f56440cb980 legacy=0x7f56440ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5642ffd700 1 -- 172.21.15.184:0/3921994179 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282935954 0 0) 0x7f56440cbdf0 con 0x7f56440cb980 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f5643fff700 1 -- 172.21.15.184:0/3921994179 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440cf8b0 con 0x7f56440b2a90 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.414+0000 7f56489e6700 1 --2- 172.21.15.184:0/3921994179 >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 0x7f56440cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f5643fff700 1 --2- 172.21.15.184:0/3921994179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440b2a90 0x7f56440cb440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56380098d0 tx=0x7f5638009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e01082f337e1341 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f5642ffd700 1 -- 172.21.15.184:0/3921994179 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638009e70 con 0x7f56440b2a90 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f5642ffd700 1 -- 172.21.15.184:0/3921994179 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638005c10 con 0x7f56440b2a90 2023-10-06T01:39:27.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f5642ffd700 1 -- 172.21.15.184:0/3921994179 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800b610 con 0x7f56440b2a90 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f564a449700 1 -- 172.21.15.184:0/3921994179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440b2a90 msgr2=0x7f56440cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f564a449700 1 --2- 172.21.15.184:0/3921994179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440b2a90 0x7f56440cb440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56380098d0 tx=0x7f5638009120 comp rx=0 tx=0).stop 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f564a449700 1 -- 172.21.15.184:0/3921994179 shutdown_connections 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f564a449700 1 --2- 172.21.15.184:0/3921994179 >> v2:172.21.15.98:3300/0 conn(0x7f56440c6610 0x7f56440cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.415+0000 7f564a449700 1 --2- 172.21.15.184:0/3921994179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440b2a90 0x7f56440cb440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.416+0000 7f564a449700 1 -- 172.21.15.184:0/3921994179 >> 172.21.15.184:0/3921994179 conn(0x7f5644099740 msgr2=0x7f564406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.416+0000 7f564a449700 1 -- 172.21.15.184:0/3921994179 shutdown_connections 2023-10-06T01:39:27.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.416+0000 7f564a449700 1 -- 172.21.15.184:0/3921994179 wait complete. 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.417+0000 7f564a449700 1 Processor -- start 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.417+0000 7f564a449700 1 -- start start 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56440b2a90 0x7f5644158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441528f0 con 0x7f56440b2a90 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5644159050 con 0x7f56440cb980 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564409b250 con 0x7f56440c6610 2023-10-06T01:39:27.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56437fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54798/0 (socket says 172.21.15.184:54798) 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56437fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38250/0 (socket says 172.21.15.184:38250) 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 -- 172.21.15.184:0/2535436596 learned_addr learned my addr 172.21.15.184:0/2535436596 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f5643fff700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56440b2a90 0x7f5644158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 -- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 msgr2=0x7f56441697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 -- 172.21.15.184:0/2535436596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56440b2a90 msgr2=0x7f5644158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56440b2a90 0x7f5644158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 -- 172.21.15.184:0/2535436596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380093c0 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56437fe700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f56489e6700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d180 tx=0x7f563c009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00dcc0 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c00f040 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56441561c0 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c015690 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.418+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644156780 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.419+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f563c015ac0 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.419+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f563c00de40 con 0x7f56440cb980 2023-10-06T01:39:27.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.419+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564409b470 con 0x7f56440cb980 2023-10-06T01:39:27.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.421+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f564409b470 con 0x7f56440cb980 2023-10-06T01:39:27.434 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:27.434+0000 7f264d609180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T01:39:27.472 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:27.471+0000 7fa495fb9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T01:39:27.505 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.504+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6c83413-b7df-4e9a-8c56-826e9ea15c07", "id": 2} v 0) v1 -- 0x7f5644064ad0 con 0x7f56440cb980 2023-10-06T01:39:27.509 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.508+0000 7f562bfff700 1 -- 172.21.15.184:0/2535436596 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6c83413-b7df-4e9a-8c56-826e9ea15c07", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f563c01a070 con 0x7f56440cb980 2023-10-06T01:39:27.509 INFO:teuthology.orchestra.run.smithi184.stdout:2 2023-10-06T01:39:27.510 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 msgr2=0x7f564409daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d180 tx=0x7f563c009a50 comp rx=0 tx=0).stop 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 shutdown_connections 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56440cb980 0x7f564409daa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56440b2a90 0x7f5644158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 --2- 172.21.15.184:0/2535436596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56440c6610 0x7f56441697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 >> 172.21.15.184:0/2535436596 conn(0x7f5644099740 msgr2=0x7f5644069970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 shutdown_connections 2023-10-06T01:39:27.511 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.509+0000 7f564a449700 1 -- 172.21.15.184:0/2535436596 wait complete. 2023-10-06T01:39:27.522 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 080f0848-851e-4596-84c3-b70d766b7b30 3 2023-10-06T01:39:27.566 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:27.565+0000 7f264d609180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T01:39:27.611 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:27.609+0000 7fa495fb9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T01:39:27.637 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.636+0000 7f0d82efd700 1 Processor -- start 2023-10-06T01:39:27.637 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.636+0000 7f0d82efd700 1 -- start start 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d82efd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d7c06ce50 0x7f0d7c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d82efd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d82efd700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c15ab90 con 0x7f0d7c06d6b0 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d82efd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c15d690 con 0x7f0d7c06ce50 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d82efd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15a5d0 con 0x7f0d7c06da80 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38260/0 (socket says 172.21.15.184:38260) 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 -- 172.21.15.184:0/3644703499 learned_addr learned my addr 172.21.15.184:0/3644703499 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d7bfff700 1 --2- 172.21.15.184:0/3644703499 >> v2:172.21.15.98:3300/0 conn(0x7f0d7c06ce50 0x7f0d7c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 -- 172.21.15.184:0/3644703499 >> v2:172.21.15.98:3300/0 conn(0x7f0d7c06ce50 msgr2=0x7f0d7c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:27.640 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 --2- 172.21.15.184:0/3644703499 >> v2:172.21.15.98:3300/0 conn(0x7f0d7c06ce50 0x7f0d7c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d7b7fe700 1 -- 172.21.15.184:0/3644703499 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387203091 0 0) 0x7f0d70001160 con 0x7f0d7c06d6b0 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 -- 172.21.15.184:0/3644703499 >> v1:172.21.15.98:6789/0 conn(0x7f0d7c06d6b0 legacy=0x7f0d7c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 -- 172.21.15.184:0/3644703499 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c15a8b0 con 0x7f0d7c06da80 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.637+0000 7f0d8149a700 1 --2- 172.21.15.184:0/3644703499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f50 tx=0x7f0d74009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21693f9a842ca32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d7b7fe700 1 -- 172.21.15.184:0/3644703499 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400e070 con 0x7f0d7c06da80 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d7b7fe700 1 -- 172.21.15.184:0/3644703499 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400b9c0 con 0x7f0d7c06da80 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d7b7fe700 1 -- 172.21.15.184:0/3644703499 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740092e0 con 0x7f0d7c06da80 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 -- 172.21.15.184:0/3644703499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 msgr2=0x7f0d7c0b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3644703499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f50 tx=0x7f0d74009840 comp rx=0 tx=0).stop 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 -- 172.21.15.184:0/3644703499 shutdown_connections 2023-10-06T01:39:27.641 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3644703499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c06da80 0x7f0d7c0b10c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.642 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3644703499 >> v2:172.21.15.98:3300/0 conn(0x7f0d7c06ce50 0x7f0d7c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.642 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 -- 172.21.15.184:0/3644703499 >> 172.21.15.184:0/3644703499 conn(0x7f0d7c06b640 msgr2=0x7f0d7c06ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.642 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 -- 172.21.15.184:0/3644703499 shutdown_connections 2023-10-06T01:39:27.642 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.638+0000 7f0d82efd700 1 -- 172.21.15.184:0/3644703499 wait complete. 2023-10-06T01:39:27.642 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:27.640+0000 7f264d609180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T01:39:27.644 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.639+0000 7f0d82efd700 1 Processor -- start 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 -- start start 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d7c06d6b0 0x7f0d7c16a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 0x7f0d7c14c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1385b0 con 0x7f0d7c06d6b0 2023-10-06T01:39:27.645 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c14e4d0 con 0x7f0d7c06da80 2023-10-06T01:39:27.646 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d82efd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c14e7b0 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.646 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d7bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.647 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d7bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54802/0 (socket says 172.21.15.184:54802) 2023-10-06T01:39:27.647 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d8149a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 0x7f0d7c14c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.647 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d7bfff700 1 -- 172.21.15.184:0/3484114812 learned_addr learned my addr 172.21.15.184:0/3484114812 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.647 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.640+0000 7f0d80c99700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d7c06d6b0 0x7f0d7c16a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 -- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 msgr2=0x7f0d7c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 -- 172.21.15.184:0/3484114812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d7c06d6b0 msgr2=0x7f0d7c16a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d7c06d6b0 0x7f0d7c16a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 -- 172.21.15.184:0/3484114812 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009ae0 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d7bfff700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:27.648 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d8149a700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 0x7f0d7c14c170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d74015000 tx=0x7f0d74018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74008b10 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74008c70 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c14ea90 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74025460 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c14f050 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0d74003040 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.641+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f0d74025d60 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.649 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.642+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c064ad0 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.650 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.644+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0d74020070 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.689 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:27.688+0000 7fa495fb9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T01:39:27.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.726+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "080f0848-851e-4596-84c3-b70d766b7b30", "id": 3} v 0) v1 -- 0x7f0d7c15d320 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.730+0000 7f0d78ff9700 1 -- 172.21.15.184:0/3484114812 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "080f0848-851e-4596-84c3-b70d766b7b30", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0d740258b0 con 0x7f0d7c0ae7d0 2023-10-06T01:39:27.731 INFO:teuthology.orchestra.run.smithi184.stdout:3 2023-10-06T01:39:27.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 msgr2=0x7f0d7c14c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 0x7f0d7c14c170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d74015000 tx=0x7f0d74018000 comp rx=0 tx=0).stop 2023-10-06T01:39:27.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 shutdown_connections 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7c0ae7d0 0x7f0d7c14c170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d7c06d6b0 0x7f0d7c16a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 --2- 172.21.15.184:0/3484114812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7c06da80 0x7f0d7c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 >> 172.21.15.184:0/3484114812 conn(0x7f0d7c06b640 msgr2=0x7f0d7c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 shutdown_connections 2023-10-06T01:39:27.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.731+0000 7f0d82efd700 1 -- 172.21.15.184:0/3484114812 wait complete. 2023-10-06T01:39:27.744 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new e213de30-3383-4cbb-9b68-1e22e089b94f 4 2023-10-06T01:39:27.824 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:27.824+0000 7f264d609180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T01:39:27.862 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 Processor -- start 2023-10-06T01:39:27.862 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 -- start start 2023-10-06T01:39:27.864 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcc4c06bff0 0x7fcc4c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.864 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc4c06a460 0x7fcc4c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.864 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c15d170 con 0x7fcc4c06a090 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c15a960 con 0x7fcc4c06bff0 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.861+0000 7fcc52296700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c15d3c0 con 0x7fcc4c06a460 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc4c06a460 0x7fcc4c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc4c06a460 0x7fcc4c15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54818/0 (socket says 172.21.15.184:54818) 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 -- 172.21.15.184:0/347246295 learned_addr learned my addr 172.21.15.184:0/347246295 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc4b7fe700 1 --2- 172.21.15.184:0/347246295 >> v2:172.21.15.98:3300/0 conn(0x7fcc4c06bff0 0x7fcc4c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 -- 172.21.15.184:0/347246295 >> v2:172.21.15.98:3300/0 conn(0x7fcc4c06bff0 msgr2=0x7fcc4c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 --2- 172.21.15.184:0/347246295 >> v2:172.21.15.98:3300/0 conn(0x7fcc4c06bff0 0x7fcc4c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 -- 172.21.15.184:0/347246295 >> v1:172.21.15.98:6789/0 conn(0x7fcc4c06a090 legacy=0x7fcc4c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.865 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 -- 172.21.15.184:0/347246295 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c15d6a0 con 0x7fcc4c06a460 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc50833700 1 --2- 172.21.15.184:0/347246295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a460 0x7fcc4c15a420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c06caf0 tx=0x7fcc44009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17667be03507a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc4affd700 1 -- 172.21.15.184:0/347246295 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4400e070 con 0x7fcc4c06a460 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc4affd700 1 -- 172.21.15.184:0/347246295 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4400ba20 con 0x7fcc4c06a460 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.862+0000 7fcc4affd700 1 -- 172.21.15.184:0/347246295 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440092e0 con 0x7fcc4c06a460 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 -- 172.21.15.184:0/347246295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a460 msgr2=0x7fcc4c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 --2- 172.21.15.184:0/347246295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a460 0x7fcc4c15a420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c06caf0 tx=0x7fcc44009840 comp rx=0 tx=0).stop 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 -- 172.21.15.184:0/347246295 shutdown_connections 2023-10-06T01:39:27.866 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 --2- 172.21.15.184:0/347246295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a460 0x7fcc4c15a420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.867 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 --2- 172.21.15.184:0/347246295 >> v2:172.21.15.98:3300/0 conn(0x7fcc4c06bff0 0x7fcc4c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.867 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 -- 172.21.15.184:0/347246295 >> 172.21.15.184:0/347246295 conn(0x7fcc4c0ce330 msgr2=0x7fcc4c0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.867 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 -- 172.21.15.184:0/347246295 shutdown_connections 2023-10-06T01:39:27.867 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.863+0000 7fcc52296700 1 -- 172.21.15.184:0/347246295 wait complete. 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.864+0000 7fcc52296700 1 Processor -- start 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.864+0000 7fcc52296700 1 -- start start 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 0x7fcc4c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c06bff0 0x7fcc4c1696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0b0b40 con 0x7fcc4c06bff0 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c2da0 con 0x7fcc4c06a090 2023-10-06T01:39:27.869 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc52296700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0ae080 con 0x7fcc4c06a460 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 0x7fcc4c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 0x7fcc4c14b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38274/0 (socket says 172.21.15.184:38274) 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4b7fe700 1 -- 172.21.15.184:0/1796327016 learned_addr learned my addr 172.21.15.184:0/1796327016 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54824/0 (socket says 172.21.15.184:54824) 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc50833700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c06bff0 0x7fcc4c1696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 -- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 msgr2=0x7fcc4c14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 0x7fcc4c14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 -- 172.21.15.184:0/1796327016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c06bff0 msgr2=0x7fcc4c1696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c06bff0 0x7fcc4c1696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.870 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 -- 172.21.15.184:0/1796327016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44009ae0 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc4bfff700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c490 tx=0x7fcc3c007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.865+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00ce40 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c011590 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c0b02a0 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c011950 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0b0860 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcc3c01c440 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.866+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fcc3c000f90 con 0x7fcc4c06a090 2023-10-06T01:39:27.871 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.867+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c0c2b70 con 0x7fcc4c06a090 2023-10-06T01:39:27.872 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.869+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fcc4c0c2b70 con 0x7fcc4c06a090 2023-10-06T01:39:27.884 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:27.883+0000 7fa495fb9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T01:39:27.952 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.951+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e213de30-3383-4cbb-9b68-1e22e089b94f", "id": 4} v 0) v1 -- 0x7fcc4c064ad0 con 0x7fcc4c06a090 2023-10-06T01:39:27.955 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.954+0000 7fcc33fff700 1 -- 172.21.15.184:0/1796327016 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e213de30-3383-4cbb-9b68-1e22e089b94f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcc3c017070 con 0x7fcc4c06a090 2023-10-06T01:39:27.955 INFO:teuthology.orchestra.run.smithi184.stdout:4 2023-10-06T01:39:27.956 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 msgr2=0x7fcc4c0adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:27.956 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c490 tx=0x7fcc3c007ec0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 shutdown_connections 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c06bff0 0x7fcc4c1696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc4c06a090 0x7fcc4c0adb40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 --2- 172.21.15.184:0/1796327016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc4c06a460 0x7fcc4c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 >> 172.21.15.184:0/1796327016 conn(0x7fcc4c0ce330 msgr2=0x7fcc4c157790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 shutdown_connections 2023-10-06T01:39:27.957 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:27.955+0000 7fcc52296700 1 -- 172.21.15.184:0/1796327016 wait complete. 2023-10-06T01:39:27.967 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new f890cdc2-7390-4a47-ac30-19ca54ed753e 5 2023-10-06T01:39:27.996 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:27.996+0000 7f264d609180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T01:39:28.064 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:28.063+0000 7fa495fb9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T01:39:28.072 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:28.072+0000 7f264d609180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T01:39:28.083 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.082+0000 7fb4a08d0700 1 Processor -- start 2023-10-06T01:39:28.083 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.082+0000 7fb4a08d0700 1 -- start start 2023-10-06T01:39:28.086 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.086 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.086 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb49c0ceae0 0x7fb49c0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c0cc9b0 con 0x7fb49c0ceae0 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c0cf4c0 con 0x7fb49c0b2660 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb4a08d0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0cbda0 con 0x7fb49c0b2a30 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49ad9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49ad9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54830/0 (socket says 172.21.15.184:54830) 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38282/0 (socket says 172.21.15.184:38282) 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49ad9d700 1 -- 172.21.15.184:0/879412471 learned_addr learned my addr 172.21.15.184:0/879412471 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49b59e700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.98:3300/0 conn(0x7fb49c0ceae0 0x7fb49c0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 -- 172.21.15.184:0/879412471 >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 msgr2=0x7fb49c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:28.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 -- 172.21.15.184:0/879412471 >> v2:172.21.15.98:3300/0 conn(0x7fb49c0ceae0 msgr2=0x7fb49c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.98:3300/0 conn(0x7fb49c0ceae0 0x7fb49c0cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 -- 172.21.15.184:0/879412471 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0cc080 con 0x7fb49c0b2a30 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49ad9d700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.083+0000 7fb49a59c700 1 --2- 172.21.15.184:0/879412471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb49c089500 tx=0x7fb48c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d063fe4be80bcc37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb499d9b700 1 -- 172.21.15.184:0/879412471 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c011630 con 0x7fb49c0b2a30 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb499d9b700 1 -- 172.21.15.184:0/879412471 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c0117b0 con 0x7fb49c0b2a30 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb499d9b700 1 -- 172.21.15.184:0/879412471 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c011b70 con 0x7fb49c0b2a30 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 -- 172.21.15.184:0/879412471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 msgr2=0x7fb49c0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/879412471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb49c089500 tx=0x7fb48c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 -- 172.21.15.184:0/879412471 shutdown_connections 2023-10-06T01:39:28.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.184:3300/0 conn(0x7fb49c0b2660 0x7fb49c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/879412471 >> v2:172.21.15.98:3300/0 conn(0x7fb49c0ceae0 0x7fb49c0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/879412471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0b2a30 0x7fb49c0cd230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 -- 172.21.15.184:0/879412471 >> 172.21.15.184:0/879412471 conn(0x7fb49c099560 msgr2=0x7fb49c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 -- 172.21.15.184:0/879412471 shutdown_connections 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.084+0000 7fb4a08d0700 1 -- 172.21.15.184:0/879412471 wait complete. 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.085+0000 7fb4a08d0700 1 Processor -- start 2023-10-06T01:39:28.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 -- start start 2023-10-06T01:39:28.091 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.091 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c0b2a30 0x7fb49c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.091 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0ceae0 0x7fb49c1466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.091 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c147ad0 con 0x7fb49c0b2a30 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1484a0 con 0x7fb49c0b2660 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb4a08d0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c146bf0 con 0x7fb49c0ceae0 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb49ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb49b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0ceae0 0x7fb49c1466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb49ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54834/0 (socket says 172.21.15.184:54834) 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb49ad9d700 1 -- 172.21.15.184:0/2487227334 learned_addr learned my addr 172.21.15.184:0/2487227334 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.086+0000 7fb49a59c700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c0b2a30 0x7fb49c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 -- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0ceae0 msgr2=0x7fb49c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0ceae0 0x7fb49c1466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 -- 172.21.15.184:0/2487227334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c0b2a30 msgr2=0x7fb49c146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c0b2a30 0x7fb49c146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 -- 172.21.15.184:0/2487227334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490009480 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb49ad9d700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb49000ec80 tx=0x7fb490009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490005a00 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490005b60 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49000c850 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c009420 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c1471a0 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb490006030 con 0x7fb49c0b2660 2023-10-06T01:39:28.093 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.087+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fb4900139f0 con 0x7fb49c0b2660 2023-10-06T01:39:28.094 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.088+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c147db0 con 0x7fb49c0b2660 2023-10-06T01:39:28.094 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.090+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb49003a6d0 con 0x7fb49c0b2660 2023-10-06T01:39:28.144 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:28.143+0000 7fa495fb9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T01:39:28.174 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.173+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f890cdc2-7390-4a47-ac30-19ca54ed753e", "id": 5} v 0) v1 -- 0x7fb49c05f270 con 0x7fb49c0b2660 2023-10-06T01:39:28.177 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.176+0000 7fb482ffd700 1 -- 172.21.15.184:0/2487227334 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f890cdc2-7390-4a47-ac30-19ca54ed753e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb490029070 con 0x7fb49c0b2660 2023-10-06T01:39:28.177 INFO:teuthology.orchestra.run.smithi184.stdout:5 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 msgr2=0x7fb49c147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb49000ec80 tx=0x7fb490009220 comp rx=0 tx=0).stop 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 shutdown_connections 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb49c0ceae0 0x7fb49c1466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb49c0b2660 0x7fb49c147f60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 --2- 172.21.15.184:0/2487227334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c0b2a30 0x7fb49c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 >> 172.21.15.184:0/2487227334 conn(0x7fb49c099560 msgr2=0x7fb49c0ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 shutdown_connections 2023-10-06T01:39:28.179 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.177+0000 7fb4a08d0700 1 -- 172.21.15.184:0/2487227334 wait complete. 2023-10-06T01:39:28.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6 6 2023-10-06T01:39:28.206 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:28.206+0000 7f264d609180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T01:39:28.287 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:28.286+0000 7fa495fb9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T01:39:28.303 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:28.302+0000 7f264d609180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T01:39:28.308 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 Processor -- start 2023-10-06T01:39:28.308 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 -- start start 2023-10-06T01:39:28.310 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fade40b2a90 0x7fade40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.310 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fade40c6610 0x7fade40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.310 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40cbdf0 con 0x7fade40cb980 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40b2e60 con 0x7fade40c6610 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.307+0000 7fadeacc3700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40cf5d0 con 0x7fade40b2a90 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fade40b2a90 0x7fade40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade3fff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fade40cb980 0x7fade40ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42558/0 (socket says 172.21.15.184:42558) 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade3fff700 1 -- 172.21.15.184:0/1992201920 learned_addr learned my addr 172.21.15.184:0/1992201920 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade9260700 1 --2- 172.21.15.184:0/1992201920 >> v2:172.21.15.98:3300/0 conn(0x7fade40c6610 0x7fade40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 -- 172.21.15.184:0/1992201920 >> v2:172.21.15.98:3300/0 conn(0x7fade40c6610 msgr2=0x7fade40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 --2- 172.21.15.184:0/1992201920 >> v2:172.21.15.98:3300/0 conn(0x7fade40c6610 0x7fade40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 -- 172.21.15.184:0/1992201920 >> v1:172.21.15.184:6789/0 conn(0x7fade40cb980 legacy=0x7fade40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade37fe700 1 -- 172.21.15.184:0/1992201920 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477220261 0 0) 0x7fade40cbdf0 con 0x7fade40cb980 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 -- 172.21.15.184:0/1992201920 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40cf8b0 con 0x7fade40b2a90 2023-10-06T01:39:28.311 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade8a5f700 1 --2- 172.21.15.184:0/1992201920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40b2a90 0x7fade40cb440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadd80098d0 tx=0x7fadd8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967fea5775e6e9b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade37fe700 1 -- 172.21.15.184:0/1992201920 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8005c10 con 0x7fade40b2a90 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade37fe700 1 -- 172.21.15.184:0/1992201920 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8005d70 con 0x7fade40b2a90 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.308+0000 7fade37fe700 1 -- 172.21.15.184:0/1992201920 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8006260 con 0x7fade40b2a90 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 -- 172.21.15.184:0/1992201920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40b2a90 msgr2=0x7fade40cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 --2- 172.21.15.184:0/1992201920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40b2a90 0x7fade40cb440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadd80098d0 tx=0x7fadd8009120 comp rx=0 tx=0).stop 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 -- 172.21.15.184:0/1992201920 shutdown_connections 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 --2- 172.21.15.184:0/1992201920 >> v2:172.21.15.98:3300/0 conn(0x7fade40c6610 0x7fade40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 --2- 172.21.15.184:0/1992201920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40b2a90 0x7fade40cb440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 -- 172.21.15.184:0/1992201920 >> 172.21.15.184:0/1992201920 conn(0x7fade4099740 msgr2=0x7fade406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 -- 172.21.15.184:0/1992201920 shutdown_connections 2023-10-06T01:39:28.312 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.309+0000 7fadeacc3700 1 -- 172.21.15.184:0/1992201920 wait complete. 2023-10-06T01:39:28.314 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.310+0000 7fadeacc3700 1 Processor -- start 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.310+0000 7fadeacc3700 1 -- start start 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fade40b2a90 0x7fade416ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade40cb980 0x7fade415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41536a0 con 0x7fade40cb980 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41445a0 con 0x7fade40c6610 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fadeacc3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4152a90 con 0x7fade40b2a90 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.315 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade8a5f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fade40b2a90 0x7fade416ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54850/0 (socket says 172.21.15.184:54850) 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 -- 172.21.15.184:0/2898945215 learned_addr learned my addr 172.21.15.184:0/2898945215 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade9260700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade40cb980 0x7fade415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 -- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fade40b2a90 msgr2=0x7fade416ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fade40b2a90 0x7fade416ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 -- 172.21.15.184:0/2898945215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade40cb980 msgr2=0x7fade415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade40cb980 0x7fade415ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 -- 172.21.15.184:0/2898945215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80093c0 con 0x7fade40c6610 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.311+0000 7fade3fff700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c5b0 tx=0x7fadd0001070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.316 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0005b30 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0005c90 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4152d70 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0007300 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade4153330 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fadd001c6c0 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fadd0005e00 con 0x7fade40c6610 2023-10-06T01:39:28.317 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.312+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade4064ad0 con 0x7fade40c6610 2023-10-06T01:39:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.314+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fadd0017070 con 0x7fade40c6610 2023-10-06T01:39:28.386 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:28.385+0000 7fa495fb9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T01:39:28.398 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.397+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6", "id": 6} v 0) v1 -- 0x7fade41537e0 con 0x7fade40c6610 2023-10-06T01:39:28.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.406+0000 7fade0ff9700 1 -- 172.21.15.184:0/2898945215 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fadd001cb50 con 0x7fade40c6610 2023-10-06T01:39:28.407 INFO:teuthology.orchestra.run.smithi184.stdout:6 2023-10-06T01:39:28.409 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 msgr2=0x7fade4144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.409 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c5b0 tx=0x7fadd0001070 comp rx=0 tx=0).stop 2023-10-06T01:39:28.409 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 shutdown_connections 2023-10-06T01:39:28.409 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade40cb980 0x7fade415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fade40b2a90 0x7fade416ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 --2- 172.21.15.184:0/2898945215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fade40c6610 0x7fade4144060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 >> 172.21.15.184:0/2898945215 conn(0x7fade4099740 msgr2=0x7fade4151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 shutdown_connections 2023-10-06T01:39:28.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.407+0000 7fadeacc3700 1 -- 172.21.15.184:0/2898945215 wait complete. 2023-10-06T01:39:28.420 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new eda3a706-f9f2-4757-89f7-65d481de0792 7 2023-10-06T01:39:28.536 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.534+0000 7f2d8eab2700 1 Processor -- start 2023-10-06T01:39:28.536 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 -- start start 2023-10-06T01:39:28.538 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d880c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.538 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d8806a570 0x7f2d8806aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.538 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d880c8b30 con 0x7f2d88156bb0 2023-10-06T01:39:28.538 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8806c1c0 con 0x7f2d8806a570 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8eab2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8806af50 con 0x7f2d88157690 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8c84e700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2d88156bb0 0x7f2d88156f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:42570/0 (socket says 172.21.15.184:42570) 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d87fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d880c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.535+0000 7f2d8c84e700 1 -- 172.21.15.184:0/1205620328 learned_addr learned my addr 172.21.15.184:0/1205620328 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d8d04f700 1 --2- 172.21.15.184:0/1205620328 >> v2:172.21.15.98:3300/0 conn(0x7f2d8806a570 0x7f2d8806aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d877fe700 1 -- 172.21.15.184:0/1205620328 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808840297 0 0) 0x7f2d880c8b30 con 0x7f2d88156bb0 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d87fff700 1 -- 172.21.15.184:0/1205620328 >> v2:172.21.15.98:3300/0 conn(0x7f2d8806a570 msgr2=0x7f2d8806aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d87fff700 1 --2- 172.21.15.184:0/1205620328 >> v2:172.21.15.98:3300/0 conn(0x7f2d8806a570 0x7f2d8806aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d87fff700 1 -- 172.21.15.184:0/1205620328 >> v1:172.21.15.184:6789/0 conn(0x7f2d88156bb0 legacy=0x7f2d88156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d87fff700 1 -- 172.21.15.184:0/1205620328 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880c8e10 con 0x7f2d88157690 2023-10-06T01:39:28.539 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d87fff700 1 --2- 172.21.15.184:0/1205620328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d880c84e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098e0 tx=0x7f2d740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd16f5c5e87ef1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d877fe700 1 -- 172.21.15.184:0/1205620328 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400b640 con 0x7f2d88157690 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d877fe700 1 -- 172.21.15.184:0/1205620328 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7400b7a0 con 0x7f2d88157690 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d877fe700 1 -- 172.21.15.184:0/1205620328 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400bb40 con 0x7f2d88157690 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.536+0000 7f2d8eab2700 1 -- 172.21.15.184:0/1205620328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 msgr2=0x7f2d880c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/1205620328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d880c84e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098e0 tx=0x7f2d740091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 -- 172.21.15.184:0/1205620328 shutdown_connections 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/1205620328 >> v2:172.21.15.98:3300/0 conn(0x7f2d8806a570 0x7f2d8806aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/1205620328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d880c84e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 -- 172.21.15.184:0/1205620328 >> 172.21.15.184:0/1205620328 conn(0x7f2d880cd1e0 msgr2=0x7f2d880cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 -- 172.21.15.184:0/1205620328 shutdown_connections 2023-10-06T01:39:28.540 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.537+0000 7f2d8eab2700 1 -- 172.21.15.184:0/1205620328 wait complete. 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.538+0000 7f2d8eab2700 1 Processor -- start 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.538+0000 7f2d8eab2700 1 -- start start 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d8806a570 0x7f2d88155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880caf60 con 0x7f2d8806a570 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8814e5e0 con 0x7f2d88156bb0 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8816a180 con 0x7f2d88157690 2023-10-06T01:39:28.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d87fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d87fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54860/0 (socket says 172.21.15.184:54860) 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d87fff700 1 -- 172.21.15.184:0/3399195481 learned_addr learned my addr 172.21.15.184:0/3399195481 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38320/0 (socket says 172.21.15.184:38320) 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8c84e700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d8806a570 0x7f2d88155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 -- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 msgr2=0x7f2d88156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 -- 172.21.15.184:0/3399195481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d8806a570 msgr2=0x7f2d88155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d8806a570 0x7f2d88155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 -- 172.21.15.184:0/3399195481 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74009420 con 0x7f2d88157690 2023-10-06T01:39:28.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d87fff700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8d04f700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009e80 tx=0x7f2d8000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000caa0 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8000f040 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8816dea0 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000ce70 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.539+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d88169b30 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.540+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2d80014690 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.540+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f2d8001dbb0 con 0x7f2d88157690 2023-10-06T01:39:28.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.540+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8816a2f0 con 0x7f2d88157690 2023-10-06T01:39:28.546 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.542+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2d8003b6d0 con 0x7f2d88157690 2023-10-06T01:39:28.625 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.624+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "eda3a706-f9f2-4757-89f7-65d481de0792", "id": 7} v 0) v1 -- 0x7f2d88064ad0 con 0x7f2d88157690 2023-10-06T01:39:28.633 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.632+0000 7f2d84ff9700 1 -- 172.21.15.184:0/3399195481 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eda3a706-f9f2-4757-89f7-65d481de0792", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d8002a070 con 0x7f2d88157690 2023-10-06T01:39:28.633 INFO:teuthology.orchestra.run.smithi184.stdout:7 2023-10-06T01:39:28.634 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 msgr2=0x7f2d8816cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.634 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009e80 tx=0x7f2d8000b040 comp rx=0 tx=0).stop 2023-10-06T01:39:28.634 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 shutdown_connections 2023-10-06T01:39:28.634 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d88157690 0x7f2d8816cfb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.635 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d8806a570 0x7f2d88155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.635 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 --2- 172.21.15.184:0/3399195481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d88156bb0 0x7f2d88156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.635 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 >> 172.21.15.184:0/3399195481 conn(0x7f2d880cd1e0 msgr2=0x7f2d8814eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.635 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 shutdown_connections 2023-10-06T01:39:28.635 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-06T01:39:28.633+0000 7f2d8eab2700 1 -- 172.21.15.184:0/3399195481 wait complete. 2023-10-06T01:39:28.645 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T01:39:28.646 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T01:39:28.648 INFO:tasks.ceph.osd.0:Started 2023-10-06T01:39:28.648 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T01:39:28.648 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T01:39:28.650 INFO:tasks.ceph.osd.1:Started 2023-10-06T01:39:28.650 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T01:39:28.650 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T01:39:28.652 INFO:tasks.ceph.osd.2:Started 2023-10-06T01:39:28.652 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T01:39:28.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T01:39:28.654 INFO:tasks.ceph.osd.3:Started 2023-10-06T01:39:28.655 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T01:39:28.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T01:39:28.657 INFO:tasks.ceph.osd.4:Started 2023-10-06T01:39:28.657 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T01:39:28.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T01:39:28.659 INFO:tasks.ceph.osd.5:Started 2023-10-06T01:39:28.659 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T01:39:28.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T01:39:28.661 INFO:tasks.ceph.osd.6:Started 2023-10-06T01:39:28.661 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T01:39:28.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T01:39:28.663 INFO:tasks.ceph.osd.7:Started 2023-10-06T01:39:28.664 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:28.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.811+0000 7fc153049700 1 Processor -- start 2023-10-06T01:39:28.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.811+0000 7fc153049700 1 -- start start 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc153049700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc14c068600 0x7fc14c068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc153049700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 0x7fc14c0aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc153049700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14c068fb0 con 0x7fc14c159690 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc153049700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c0a9fd0 con 0x7fc14c068600 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc153049700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0abfe0 con 0x7fc14c0aa460 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc14c068600 0x7fc14c068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc150de5700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc14c159690 0x7fc14c0ac480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38124/0 (socket says 172.21.15.98:38124) 2023-10-06T01:39:28.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc150de5700 1 -- 172.21.15.98:0/1331609265 learned_addr learned my addr 172.21.15.98:0/1331609265 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc1515e6700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 0x7fc14c0aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 -- 172.21.15.98:0/1331609265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 msgr2=0x7fc14c0aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 0x7fc14c0aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 -- 172.21.15.98:0/1331609265 >> v1:172.21.15.98:6789/0 conn(0x7fc14c159690 legacy=0x7fc14c0ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 -- 172.21.15.98:0/1331609265 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0abd00 con 0x7fc14c068600 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14b7fe700 1 -- 172.21.15.98:0/1331609265 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378682100 0 0) 0x7fc14c068fb0 con 0x7fc14c159690 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc1515e6700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 0x7fc14c0aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14bfff700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c068600 0x7fc14c068a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098e0 tx=0x7fc1380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d56af6236ed2f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14b7fe700 1 -- 172.21.15.98:0/1331609265 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138009e80 con 0x7fc14c068600 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14b7fe700 1 -- 172.21.15.98:0/1331609265 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13800b860 con 0x7fc14c068600 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.812+0000 7fc14b7fe700 1 -- 172.21.15.98:0/1331609265 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800bbe0 con 0x7fc14c068600 2023-10-06T01:39:28.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 -- 172.21.15.98:0/1331609265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c068600 msgr2=0x7fc14c068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c068600 0x7fc14c068a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098e0 tx=0x7fc1380091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 -- 172.21.15.98:0/1331609265 shutdown_connections 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c0aa460 0x7fc14c0aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 --2- 172.21.15.98:0/1331609265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c068600 0x7fc14c068a70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 -- 172.21.15.98:0/1331609265 >> 172.21.15.98:0/1331609265 conn(0x7fc14c06de80 msgr2=0x7fc14c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 -- 172.21.15.98:0/1331609265 shutdown_connections 2023-10-06T01:39:28.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.813+0000 7fc153049700 1 -- 172.21.15.98:0/1331609265 wait complete. 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.814+0000 7fc153049700 1 Processor -- start 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.814+0000 7fc153049700 1 -- start start 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c068600 0x7fc14c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14c159690 0x7fc14c06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:28.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c15ee40 con 0x7fc14c0aa460 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c14f370 con 0x7fc14c159690 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc153049700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c15e230 con 0x7fc14c068600 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44348/0 (socket says 172.21.15.98:44348) 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 -- 172.21.15.98:0/1215603628 learned_addr learned my addr 172.21.15.98:0/1215603628 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc150de5700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c068600 0x7fc14c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc1515e6700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14c159690 0x7fc14c06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 -- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c068600 msgr2=0x7fc14c14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c068600 0x7fc14c14ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 -- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14c159690 msgr2=0x7fc14c06a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14c159690 0x7fc14c06a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.815+0000 7fc14bfff700 1 -- 172.21.15.98:0/1215603628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138009420 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc14bfff700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc14c06d2a0 tx=0x7fc138017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800b9d0 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13800bb30 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138011650 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c15e510 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c15ead0 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc138011d00 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.816+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc13802ee40 con 0x7fc14c0aa460 2023-10-06T01:39:28.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.817+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c064ad0 con 0x7fc14c0aa460 2023-10-06T01:39:28.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.819+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc138033060 con 0x7fc14c0aa460 2023-10-06T01:39:28.864 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:28.863+0000 7f264d609180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T01:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.902+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc14c04bb80 con 0x7fc14c0aa460 2023-10-06T01:39:28.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.902+0000 7fc148ff9700 1 -- 172.21.15.98:0/1215603628 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc13803c090 con 0x7fc14c0aa460 2023-10-06T01:39:28.903 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:28.903 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":10,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:28.625965+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:28.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 msgr2=0x7fc14c069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:28.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc14c06d2a0 tx=0x7fc138017000 comp rx=0 tx=0).stop 2023-10-06T01:39:28.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 shutdown_connections 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14c159690 0x7fc14c06a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc14c068600 0x7fc14c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 --2- 172.21.15.98:0/1215603628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14c0aa460 0x7fc14c069b90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 >> 172.21.15.98:0/1215603628 conn(0x7fc14c06de80 msgr2=0x7fc14c151910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 shutdown_connections 2023-10-06T01:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:28.903+0000 7fc153049700 1 -- 172.21.15.98:0/1215603628 wait complete. 2023-10-06T01:39:28.915 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T01:39:28.916 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T01:39:28.934 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:28.933+0000 7fa495fb9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T01:39:28.944 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:28.943+0000 7f264d609180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T01:39:29.011 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:29.009+0000 7fa495fb9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T01:39:29.021 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:29.021+0000 7f264d609180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T01:39:29.089 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:29.088+0000 7fa495fb9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T01:39:29.250 INFO:tasks.ceph.osd.0.smithi098.stderr:2023-10-06T01:39:29.249+0000 7f31e10ee540 -1 Falling back to public interface 2023-10-06T01:39:29.253 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-06T01:39:29.252+0000 7f3a72985540 -1 Falling back to public interface 2023-10-06T01:39:29.253 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-06T01:39:29.252+0000 7f42e2daf540 -1 Falling back to public interface 2023-10-06T01:39:29.257 INFO:tasks.ceph.osd.1.smithi098.stderr:2023-10-06T01:39:29.256+0000 7f3a9c7bf540 -1 Falling back to public interface 2023-10-06T01:39:29.257 INFO:tasks.ceph.osd.3.smithi098.stderr:2023-10-06T01:39:29.256+0000 7fe3fb13b540 -1 Falling back to public interface 2023-10-06T01:39:29.257 INFO:tasks.ceph.osd.3.smithi098.stderr:2023-10-06T01:39:29.256+0000 7fe3f5564700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-10-06T01:39:29.261 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-06T01:39:29.260+0000 7fcaa87fd540 -1 Falling back to public interface 2023-10-06T01:39:29.265 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-06T01:39:29.265+0000 7fd181a8b540 -1 Falling back to public interface 2023-10-06T01:39:29.274 INFO:tasks.ceph.osd.2.smithi098.stderr:2023-10-06T01:39:29.274+0000 7f25e122c540 -1 Falling back to public interface 2023-10-06T01:39:29.610 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:29.610+0000 7f264d609180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T01:39:29.679 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:29.678+0000 7fa495fb9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T01:39:29.721 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:29.749 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:29.749+0000 7f264d609180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T01:39:29.816 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:29.814+0000 7fa495fb9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T01:39:29.831 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.830+0000 7fc458dc4700 1 Processor -- start 2023-10-06T01:39:29.831 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.830+0000 7fc458dc4700 1 -- start start 2023-10-06T01:39:29.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc458dc4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4540cc490 0x7fc45406ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:29.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc458dc4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 0x7fc454158540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:29.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc458dc4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45406b860 con 0x7fc454156350 2023-10-06T01:39:29.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc458dc4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc454157730 con 0x7fc4540cc490 2023-10-06T01:39:29.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc458dc4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406bb40 con 0x7fc45406b070 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc45259c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc454156350 0x7fc4540cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38128/0 (socket says 172.21.15.98:38128) 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc45259c700 1 -- 172.21.15.98:0/1438266142 learned_addr learned my addr 172.21.15.98:0/1438266142 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1438266142 >> v2:172.21.15.184:3300/0 conn(0x7fc4540cc490 0x7fc45406ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc452d9d700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 0x7fc454158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240478073 0 0) 0x7fc45406b860 con 0x7fc454156350 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc440003570 con 0x7fc454156350 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.831+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758919151 0 0) 0x7fc440003570 con 0x7fc454156350 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc44bfff700 1 -- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 msgr2=0x7fc454158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 0x7fc454158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc44bfff700 1 -- 172.21.15.98:0/1438266142 >> v1:172.21.15.98:6789/0 conn(0x7fc454156350 legacy=0x7fc4540cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc44bfff700 1 -- 172.21.15.98:0/1438266142 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454157a60 con 0x7fc4540cc490 2023-10-06T01:39:29.834 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc452d9d700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 0x7fc454158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4540cc490 0x7fc45406ab30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0098e0 tx=0x7fc43c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3c3e88896ca8c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444001470 con 0x7fc4540cc490 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c011420 con 0x7fc4540cc490 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc451d9b700 1 -- 172.21.15.98:0/1438266142 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0117c0 con 0x7fc4540cc490 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 -- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4540cc490 msgr2=0x7fc45406ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4540cc490 0x7fc45406ab30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0098e0 tx=0x7fc43c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 -- 172.21.15.98:0/1438266142 shutdown_connections 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc45406b070 0x7fc454158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1438266142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4540cc490 0x7fc45406ab30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 -- 172.21.15.98:0/1438266142 >> 172.21.15.98:0/1438266142 conn(0x7fc4540675f0 msgr2=0x7fc4540679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:29.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 -- 172.21.15.98:0/1438266142 shutdown_connections 2023-10-06T01:39:29.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.832+0000 7fc458dc4700 1 -- 172.21.15.98:0/1438266142 wait complete. 2023-10-06T01:39:29.838 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.833+0000 7fc458dc4700 1 Processor -- start 2023-10-06T01:39:29.838 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 -- start start 2023-10-06T01:39:29.838 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 0x7fc454152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:29.838 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:29.838 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc454156350 0x7fc45415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541684e0 con 0x7fc4540cc490 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4540ca130 con 0x7fc45406b070 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc458dc4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc454168050 con 0x7fc454156350 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc44bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc44bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44356/0 (socket says 172.21.15.98:44356) 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc44bfff700 1 -- 172.21.15.98:0/1729257392 learned_addr learned my addr 172.21.15.98:0/1729257392 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc45259c700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 0x7fc454152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.834+0000 7fc452d9d700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc454156350 0x7fc45415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 -- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc454156350 msgr2=0x7fc45415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc454156350 0x7fc45415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.839 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 -- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 msgr2=0x7fc454152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 0x7fc454152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 -- 172.21.15.98:0/1729257392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c009a20 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc45259c700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 0x7fc454152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44bfff700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc454069fc0 tx=0x7fc43c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c01cba0 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c01cd00 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c009420 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c01a6a0 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc454167a00 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc43c01ad50 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.835+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc43c024af0 con 0x7fc4540cc490 2023-10-06T01:39:29.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.836+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc454064ad0 con 0x7fc4540cc490 2023-10-06T01:39:29.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.838+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc43c029070 con 0x7fc4540cc490 2023-10-06T01:39:29.853 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:29.852+0000 7f264d609180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T01:39:29.907 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.906+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc454157d40 con 0x7fc4540cc490 2023-10-06T01:39:29.907 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.907+0000 7fc44a7fc700 1 -- 172.21.15.98:0/1729257392 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc43c024410 con 0x7fc4540cc490 2023-10-06T01:39:29.908 INFO:teuthology.misc.health.smithi098.stdout: 2023-10-06T01:39:29.908 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":10,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:28.625965+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 msgr2=0x7fc454143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc454069fc0 tx=0x7fc43c016000 comp rx=0 tx=0).stop 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 shutdown_connections 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc454156350 0x7fc45415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc45406b070 0x7fc454152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 --2- 172.21.15.98:0/1729257392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4540cc490 0x7fc454143d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:29.909 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 >> 172.21.15.98:0/1729257392 conn(0x7fc4540675f0 msgr2=0x7fc4540679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:29.910 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 shutdown_connections 2023-10-06T01:39:29.910 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:29.908+0000 7fc458dc4700 1 -- 172.21.15.98:0/1729257392 wait complete. 2023-10-06T01:39:29.915 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:29.914+0000 7fa495fb9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T01:39:29.920 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T01:39:30.104 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:30.103+0000 7f264d609180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T01:39:30.166 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:30.165+0000 7fa495fb9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T01:39:30.180 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:30.180+0000 7f264d609180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T01:39:30.243 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:30.242+0000 7fa495fb9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T01:39:31.015 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.014+0000 7f264d609180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T01:39:31.071 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.070+0000 7fa495fb9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T01:39:31.088 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.088+0000 7f264d609180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T01:39:31.144 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.143+0000 7fa495fb9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T01:39:31.162 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.162+0000 7f264d609180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T01:39:31.220 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.219+0000 7fa495fb9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T01:39:31.294 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.294+0000 7f264d609180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T01:39:31.353 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.352+0000 7fa495fb9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T01:39:31.379 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.378+0000 7f264d609180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T01:39:31.437 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.436+0000 7fa495fb9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T01:39:31.533 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.531+0000 7f264d609180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T01:39:31.590 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.589+0000 7fa495fb9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T01:39:31.607 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.607+0000 7f264d609180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T01:39:31.665 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.664+0000 7fa495fb9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T01:39:31.683 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.682+0000 7f264d609180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T01:39:31.743 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.742+0000 7fa495fb9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T01:39:31.764 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:31.764+0000 7f264d609180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T01:39:31.822 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-06T01:39:31.821+0000 7fa495fb9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T01:39:32.750 INFO:tasks.ceph.osd.3.smithi098.stderr:2023-10-06T01:39:32.749+0000 7fe3fb13b540 -1 osd.3 0 log_to_monitors true 2023-10-06T01:39:32.751 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-06T01:39:32.750+0000 7f3a72985540 -1 osd.6 0 log_to_monitors true 2023-10-06T01:39:32.761 INFO:tasks.ceph.osd.1.smithi098.stderr:2023-10-06T01:39:32.760+0000 7f3a9c7bf540 -1 osd.1 0 log_to_monitors true 2023-10-06T01:39:32.776 INFO:tasks.ceph.osd.2.smithi098.stderr:2023-10-06T01:39:32.775+0000 7f25e122c540 -1 osd.2 0 log_to_monitors true 2023-10-06T01:39:32.779 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-06T01:39:32.779+0000 7f42e2daf540 -1 osd.5 0 log_to_monitors true 2023-10-06T01:39:32.787 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:32.786+0000 7f2606a0d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T01:39:32.788 INFO:tasks.ceph.mgr.y.smithi098.stderr:2023-10-06T01:39:32.788+0000 7f2606a0d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T01:39:32.790 INFO:tasks.ceph.osd.0.smithi098.stderr:2023-10-06T01:39:32.790+0000 7f31e10ee540 -1 osd.0 0 log_to_monitors true 2023-10-06T01:39:32.820 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-06T01:39:32.819+0000 7fcaa87fd540 -1 osd.7 0 log_to_monitors true 2023-10-06T01:39:32.847 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-06T01:39:32.846+0000 7fd181a8b540 -1 osd.4 0 log_to_monitors true 2023-10-06T01:39:36.008 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-06T01:39:36.006+0000 7fd179eb0700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T01:39:36.013 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-06T01:39:36.013+0000 7fd1714a4700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.016 INFO:tasks.ceph.osd.0.smithi098.stderr:2023-10-06T01:39:36.014+0000 7f31d7d10700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T01:39:36.019 INFO:tasks.ceph.osd.2.smithi098.stderr:2023-10-06T01:39:36.017+0000 7f25d7e4e700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T01:39:36.025 INFO:tasks.ceph.osd.0.smithi098.stderr:2023-10-06T01:39:36.024+0000 7f31d0b07700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.029 INFO:tasks.ceph.osd.2.smithi098.stderr:2023-10-06T01:39:36.028+0000 7f25d0c45700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.055 INFO:tasks.ceph.osd.3.smithi098.stderr:2023-10-06T01:39:36.054+0000 7fe3f1d5d700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T01:39:36.060 INFO:tasks.ceph.osd.3.smithi098.stderr:2023-10-06T01:39:36.060+0000 7fe3eab54700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.075 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-06T01:39:36.074+0000 7f3a695a7700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T01:39:36.081 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-06T01:39:36.079+0000 7f3a6239e700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.088 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-06T01:39:36.087+0000 7fca9f41f700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T01:39:36.094 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-06T01:39:36.093+0000 7fca98216700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.096 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-06T01:39:36.096+0000 7f42db1d4700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T01:39:36.100 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-06T01:39:36.099+0000 7f42d27c8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.264 INFO:tasks.ceph.osd.1.smithi098.stderr:2023-10-06T01:39:36.263+0000 7f3a933e1700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T01:39:36.268 INFO:tasks.ceph.osd.1.smithi098.stderr:2023-10-06T01:39:36.267+0000 7f3a8c1d8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T01:39:36.726 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:36.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.831+0000 7f28cb87f700 1 Processor -- start 2023-10-06T01:39:36.833 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.831+0000 7f28cb87f700 1 -- start start 2023-10-06T01:39:36.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28cb87f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f28c40cb1f0 0x7f28c40c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:36.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28cb87f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c40c77f0 con 0x7f28c4142cc0 2023-10-06T01:39:36.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28cb87f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c406b1c0 con 0x7f28c40cad80 2023-10-06T01:39:36.835 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28cb87f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c406b4a0 con 0x7f28c40cb1f0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28c961b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f28c4142cc0 0x7f28c406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33604/0 (socket says 172.21.15.98:33604) 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28c961b700 1 -- 172.21.15.98:0/3895951719 learned_addr learned my addr 172.21.15.98:0/3895951719 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.832+0000 7f28c9e1c700 1 --2- 172.21.15.98:0/3895951719 >> v2:172.21.15.98:3300/0 conn(0x7f28c40cb1f0 0x7f28c40c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28bbfff700 1 -- 172.21.15.98:0/3895951719 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079275120 0 0) 0x7f28c40c77f0 con 0x7f28c4142cc0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28bbfff700 1 -- 172.21.15.98:0/3895951719 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b0003570 con 0x7f28c4142cc0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28c9e1c700 1 -- 172.21.15.98:0/3895951719 >> v1:172.21.15.184:6789/0 conn(0x7f28c40cad80 legacy=0x7f28c40c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28c9e1c700 1 -- 172.21.15.98:0/3895951719 >> v1:172.21.15.98:6789/0 conn(0x7f28c4142cc0 legacy=0x7f28c406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28c9e1c700 1 -- 172.21.15.98:0/3895951719 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c406b780 con 0x7f28c40cb1f0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28c9e1c700 1 --2- 172.21.15.98:0/3895951719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cb1f0 0x7f28c40c8390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009e60 tx=0x7f28bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50545742610999b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28bbfff700 1 -- 172.21.15.98:0/3895951719 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00e070 con 0x7f28c40cb1f0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28bbfff700 1 -- 172.21.15.98:0/3895951719 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc00bb60 con 0x7f28c40cb1f0 2023-10-06T01:39:36.836 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28bbfff700 1 -- 172.21.15.98:0/3895951719 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc012430 con 0x7f28c40cb1f0 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28cb87f700 1 -- 172.21.15.98:0/3895951719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cb1f0 msgr2=0x7f28c40c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.833+0000 7f28cb87f700 1 --2- 172.21.15.98:0/3895951719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cb1f0 0x7f28c40c8390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009e60 tx=0x7f28bc009780 comp rx=0 tx=0).stop 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.834+0000 7f28cb87f700 1 -- 172.21.15.98:0/3895951719 shutdown_connections 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.834+0000 7f28cb87f700 1 --2- 172.21.15.98:0/3895951719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cb1f0 0x7f28c40c8390 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.834+0000 7f28cb87f700 1 -- 172.21.15.98:0/3895951719 >> 172.21.15.98:0/3895951719 conn(0x7f28c40675f0 msgr2=0x7f28c40679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.834+0000 7f28cb87f700 1 -- 172.21.15.98:0/3895951719 shutdown_connections 2023-10-06T01:39:36.837 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.834+0000 7f28cb87f700 1 -- 172.21.15.98:0/3895951719 wait complete. 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.835+0000 7f28cb87f700 1 Processor -- start 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.835+0000 7f28cb87f700 1 -- start start 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28c40cb1f0 0x7f28c413bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 0x7f28c413c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c413d760 con 0x7f28c40cad80 2023-10-06T01:39:36.840 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c416cb10 con 0x7f28c4142cc0 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c413cb50 con 0x7f28c40cb1f0 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59216/0 (socket says 172.21.15.98:59216) 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 -- 172.21.15.98:0/1207706525 learned_addr learned my addr 172.21.15.98:0/1207706525 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c8e1a700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28c40cb1f0 0x7f28c413bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c9e1c700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 0x7f28c413c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 -- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28c40cb1f0 msgr2=0x7f28c413bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28c40cb1f0 0x7f28c413bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 -- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 msgr2=0x7f28c413c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 0x7f28c413c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 -- 172.21.15.98:0/1207706525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009a20 con 0x7f28c40cad80 2023-10-06T01:39:36.841 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c9e1c700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 0x7f28c413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28c961b700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c590 tx=0x7f28c0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005b30 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0005cb0 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c413ce30 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007320 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.836+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c413d3f0 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.837+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4064ad0 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.838+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f28c001b3b0 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.838+0000 7f28b97fa700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 0x7f28a8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.838+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f28c0082ca0 con 0x7f28c40cad80 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.838+0000 7f28c8e1a700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 0x7f28a8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:36.842 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.838+0000 7f28c8e1a700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 0x7f28a8069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28c413dba0 tx=0x7f28b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:36.844 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:36.842+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28c0050d80 con 0x7f28c40cad80 2023-10-06T01:39:37.029 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.027+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c406ba60 con 0x7f28c40cad80 2023-10-06T01:39:37.029 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.028+0000 7f28b97fa700 1 -- 172.21.15.98:0/1207706525 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10492 (secure 0 0 0) 0x7f28c0017070 con 0x7f28c40cad80 2023-10-06T01:39:37.029 INFO:teuthology.misc.health.smithi098.stdout: 2023-10-06T01:39:37.030 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":13,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:37.022322+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:37.032 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 msgr2=0x7f28a8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.032 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 0x7f28a8069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28c413dba0 tx=0x7f28b4009040 comp rx=0 tx=0).stop 2023-10-06T01:39:37.032 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 msgr2=0x7f28c416c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c590 tx=0x7f28c0007e00 comp rx=0 tx=0).stop 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 shutdown_connections 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f28a8066ea0 0x7f28a8069350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28c4142cc0 0x7f28c413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28c40cad80 0x7f28c416c5d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.033 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 --2- 172.21.15.98:0/1207706525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28c40cb1f0 0x7f28c413bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.034 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 >> 172.21.15.98:0/1207706525 conn(0x7f28c40675f0 msgr2=0x7f28c40c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:37.034 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 shutdown_connections 2023-10-06T01:39:37.034 INFO:teuthology.misc.health.smithi098.stderr:2023-10-06T01:39:37.030+0000 7f28cb87f700 1 -- 172.21.15.98:0/1207706525 wait complete. 2023-10-06T01:39:37.045 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T01:39:37.045 INFO:tasks.ceph:Creating RBD pool 2023-10-06T01:39:37.045 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T01:39:37.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.158+0000 7f1f64132700 1 Processor -- start 2023-10-06T01:39:37.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 -- start start 2023-10-06T01:39:37.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cbed0 con 0x7f1f5c0cba70 2023-10-06T01:39:37.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0cf820 con 0x7f1f5c159710 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.159+0000 7f1f64132700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0cfb00 con 0x7f1f5c159340 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f616cd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f616cd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:38930/0 (socket says 172.21.15.98:38930) 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:34324/0 (socket says 172.21.15.98:34324) 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f626cf700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f1f5c0cba70 0x7f1f5c0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:42536/0 (socket says 172.21.15.98:42536) 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f616cd700 1 -- 172.21.15.98:0/2746749465 learned_addr learned my addr 172.21.15.98:0/2746749465 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 -- 172.21.15.98:0/2746749465 >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 msgr2=0x7f1f5c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 --2- 172.21.15.98:0/2746749465 >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 -- 172.21.15.98:0/2746749465 >> v1:172.21.15.184:6789/0 conn(0x7f1f5c0cba70 legacy=0x7f1f5c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 -- 172.21.15.98:0/2746749465 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0cfde0 con 0x7f1f5c159340 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f60ecc700 1 -- 172.21.15.98:0/2746749465 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052327663 0 0) 0x7f1f54001490 con 0x7f1f5c0cba70 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f616cd700 1 --2- 172.21.15.98:0/2746749465 >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.160+0000 7f1f61ece700 1 --2- 172.21.15.98:0/2746749465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098d0 tx=0x7f1f58009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e0c541abeb8e4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f60ecc700 1 -- 172.21.15.98:0/2746749465 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58005c10 con 0x7f1f5c159340 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f60ecc700 1 -- 172.21.15.98:0/2746749465 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58005d70 con 0x7f1f5c159340 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f60ecc700 1 -- 172.21.15.98:0/2746749465 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58006280 con 0x7f1f5c159340 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 -- 172.21.15.98:0/2746749465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 msgr2=0x7f1f5c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 --2- 172.21.15.98:0/2746749465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098d0 tx=0x7f1f58009120 comp rx=0 tx=0).stop 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 -- 172.21.15.98:0/2746749465 shutdown_connections 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 --2- 172.21.15.98:0/2746749465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c159340 0x7f1f5c0af4d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 --2- 172.21.15.98:0/2746749465 >> v2:172.21.15.184:3300/0 conn(0x7f1f5c159710 0x7f1f5c0cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 -- 172.21.15.98:0/2746749465 >> 172.21.15.98:0/2746749465 conn(0x7f1f5c099720 msgr2=0x7f1f5c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:37.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 -- 172.21.15.98:0/2746749465 shutdown_connections 2023-10-06T01:39:37.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.161+0000 7f1f64132700 1 -- 172.21.15.98:0/2746749465 wait complete. 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.162+0000 7f1f64132700 1 Processor -- start 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.162+0000 7f1f64132700 1 -- start start 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c0cba70 0x7f1f5c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f5c159710 0x7f1f5c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b3d00 con 0x7f1f5c159340 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c16b320 con 0x7f1f5c159710 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f64132700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c14f600 con 0x7f1f5c0cba70 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59228/0 (socket says 172.21.15.98:59228) 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 -- 172.21.15.98:0/3056075985 learned_addr learned my addr 172.21.15.98:0/3056075985 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f61ece700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c0cba70 0x7f1f5c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f626cf700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f5c159710 0x7f1f5c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 -- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c0cba70 msgr2=0x7f1f5c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c0cba70 0x7f1f5c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 -- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f5c159710 msgr2=0x7f1f5c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f5c159710 0x7f1f5c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 -- 172.21.15.98:0/3056075985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0094f0 con 0x7f1f5c159340 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.163+0000 7f1f616cd700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00ec50 tx=0x7f1f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.164+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00c850 con 0x7f1f5c159340 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.164+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00c9b0 con 0x7f1f5c159340 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.164+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f580093c0 con 0x7f1f5c159340 2023-10-06T01:39:37.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.164+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00cd30 con 0x7f1f5c159340 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.164+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c157a40 con 0x7f1f5c159340 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.165+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c064ad0 con 0x7f1f5c159340 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.165+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f1f4c013680 con 0x7f1f5c159340 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.165+0000 7f1f51ffb700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 0x7f1f40069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.166+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f1f4c084190 con 0x7f1f5c159340 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.166+0000 7f1f61ece700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 0x7f1f40069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.166+0000 7f1f61ece700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 0x7f1f40069350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009160 tx=0x7f1f58016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.170+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f4c0519b0 con 0x7f1f5c159340 2023-10-06T01:39:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:37.353+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f1f5c0c6cb0 con 0x7f1f5c159340 2023-10-06T01:39:38.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.282+0000 7f1f51ffb700 1 -- 172.21.15.98:0/3056075985 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1f4c054fd0 con 0x7f1f5c159340 2023-10-06T01:39:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 msgr2=0x7f1f40069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 0x7f1f40069350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009160 tx=0x7f1f58016000 comp rx=0 tx=0).stop 2023-10-06T01:39:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 msgr2=0x7f1f5c0b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00ec50 tx=0x7f1f4c009250 comp rx=0 tx=0).stop 2023-10-06T01:39:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 shutdown_connections 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1f40066ea0 0x7f1f40069350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f5c159710 0x7f1f5c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f5c0cba70 0x7f1f5c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 --2- 172.21.15.98:0/3056075985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f5c159340 0x7f1f5c0b3280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 >> 172.21.15.98:0/3056075985 conn(0x7f1f5c099720 msgr2=0x7f1f5c068330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 shutdown_connections 2023-10-06T01:39:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.286+0000 7f1f64132700 1 -- 172.21.15.98:0/3056075985 wait complete. 2023-10-06T01:39:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'rbd' created 2023-10-06T01:39:38.302 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T01:39:38.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.421+0000 7fa5bfc88700 1 Processor -- start 2023-10-06T01:39:38.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.421+0000 7fa5bfc88700 1 -- start start 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5bfc88700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5b80677b0 0x7fa5b8067b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5bfc88700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5b815d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5bfc88700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b80cc740 con 0x7fa5b815da70 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5bfc88700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b80cbb30 con 0x7fa5b80677b0 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5bfc88700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cbe10 con 0x7fa5b80ccbd0 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5be225700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa5b815da70 0x7fa5b80cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33616/0 (socket says 172.21.15.98:33616) 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.422+0000 7fa5be225700 1 -- 172.21.15.98:0/3115133565 learned_addr learned my addr 172.21.15.98:0/3115133565 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bda24700 1 --2- 172.21.15.98:0/3115133565 >> v2:172.21.15.184:3300/0 conn(0x7fa5b80677b0 0x7fa5b8067b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5b815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606782662 0 0) 0x7fa5b80cc740 con 0x7fa5b815da70 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a4003570 con 0x7fa5b815da70 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15236705 0 0) 0x7fa5a4003570 con 0x7fa5b815da70 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 msgr2=0x7fa5b815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5b815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 >> v2:172.21.15.184:3300/0 conn(0x7fa5b80677b0 msgr2=0x7fa5b8067b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 --2- 172.21.15.98:0/3115133565 >> v2:172.21.15.184:3300/0 conn(0x7fa5b80677b0 0x7fa5b8067b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80cc0f0 con 0x7fa5b815da70 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fa5b0001470 con 0x7fa5b815da70 2023-10-06T01:39:38.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bda24700 1 --2- 172.21.15.98:0/3115133565 >> v2:172.21.15.184:3300/0 conn(0x7fa5b80677b0 0x7fa5b8067b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5b815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 >> v1:172.21.15.98:6789/0 conn(0x7fa5b815da70 legacy=0x7fa5b80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5a4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a4003fa0 0x7fa5a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a4004d50 0x7fa5a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cbb30 con 0x7fa5a4003fa0 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cbe10 con 0x7fa5a4004d50 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a400ba60 con 0x7fa5b80ccbd0 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a4003fa0 0x7fa5a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bda24700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5a4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5be225700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a4004d50 0x7fa5a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 -- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 msgr2=0x7fa5a4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5b80ccbd0 0x7fa5a4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 -- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a4004d50 msgr2=0x7fa5a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:38.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a4004d50 0x7fa5a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5bd223700 1 -- 172.21.15.98:0/3115133565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009420 con 0x7fa5a4003fa0 2023-10-06T01:39:38.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.423+0000 7fa5be225700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a4004d50 0x7fa5a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:38.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bd223700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a4003fa0 0x7fa5a4004810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009df0 tx=0x7fa5a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5952e3313ecd35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:38.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8009740 con 0x7fa5a4003fa0 2023-10-06T01:39:38.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800e920 con 0x7fa5a4003fa0 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bca22700 1 -- 172.21.15.98:0/3115133565 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800ece0 con 0x7fa5a4003fa0 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 -- 172.21.15.98:0/3115133565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a4003fa0 msgr2=0x7fa5a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/3115133565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a4003fa0 0x7fa5a4004810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009df0 tx=0x7fa5a80091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5be225700 1 -- 172.21.15.98:0/3115133565 reap_dead start 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 -- 172.21.15.98:0/3115133565 shutdown_connections 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 -- 172.21.15.98:0/3115133565 >> 172.21.15.98:0/3115133565 conn(0x7fa5b80cdc00 msgr2=0x7fa5b80c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 -- 172.21.15.98:0/3115133565 shutdown_connections 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.424+0000 7fa5bfc88700 1 -- 172.21.15.98:0/3115133565 wait complete. 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.425+0000 7fa5bfc88700 1 Processor -- start 2023-10-06T01:39:38.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.425+0000 7fa5bfc88700 1 -- start start 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a4003fa0 0x7fa5b80c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5b8160780 0x7fa5b806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8161020 con 0x7fa5b806b870 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815e930 con 0x7fa5b8160780 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bfc88700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815f170 con 0x7fa5a4003fa0 2023-10-06T01:39:38.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59236/0 (socket says 172.21.15.98:59236) 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 -- 172.21.15.98:0/1660327471 learned_addr learned my addr 172.21.15.98:0/1660327471 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bd223700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5b8160780 0x7fa5b806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5bda24700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a4003fa0 0x7fa5b80c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 -- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a4003fa0 msgr2=0x7fa5b80c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a4003fa0 0x7fa5b80c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 -- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5b8160780 msgr2=0x7fa5b806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5b8160780 0x7fa5b806b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 -- 172.21.15.98:0/1660327471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80c6e60 con 0x7fa5b806b870 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.426+0000 7fa5be225700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5b81615d0 tx=0x7fa5b80680c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:38.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.427+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0010520 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.427+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b00106a0 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.427+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b8160d40 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.427+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0010a60 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.427+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b815ee00 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.428+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b815f2e0 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.428+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa5b00133e0 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.429+0000 7fa5adffb700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 0x7fa59c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.429+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5184+0+0 (secure 0 0 0) 0x7fa5b0083b70 con 0x7fa5b806b870 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.429+0000 7fa5bda24700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 0x7fa59c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.429+0000 7fa5bda24700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 0x7fa59c069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40094d0 tx=0x7fa5b400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:38.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.433+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b0051050 con 0x7fa5b806b870 2023-10-06T01:39:38.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:38.616+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa5b8064ad0 con 0x7fa5b806b870 2023-10-06T01:39:39.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.285+0000 7fa5adffb700 1 -- 172.21.15.98:0/1660327471 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa5b0019020 con 0x7fa5b806b870 2023-10-06T01:39:39.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.288+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 msgr2=0x7fa59c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.288+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 0x7fa59c069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40094d0 tx=0x7fa5b400b040 comp rx=0 tx=0).stop 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 msgr2=0x7fa5b806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5b81615d0 tx=0x7fa5b80680c0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 shutdown_connections 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa59c066ea0 0x7fa59c069350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5b806b870 0x7fa5b806bd10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a4003fa0 0x7fa5b80c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 --2- 172.21.15.98:0/1660327471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5b8160780 0x7fa5b806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 >> 172.21.15.98:0/1660327471 conn(0x7fa5b80cdc00 msgr2=0x7fa5b806ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:39.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 shutdown_connections 2023-10-06T01:39:39.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.289+0000 7fa5bfc88700 1 -- 172.21.15.98:0/1660327471 wait complete. 2023-10-06T01:39:39.292 INFO:teuthology.orchestra.run.smithi098.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T01:39:39.304 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T01:39:39.304 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T01:39:39.305 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T01:39:39.306 INFO:tasks.ceph.mds.a:Started 2023-10-06T01:39:39.307 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T01:39:39.307 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T01:39:39.308 INFO:tasks.ceph.mds.c:Started 2023-10-06T01:39:39.309 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T01:39:39.309 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T01:39:39.311 INFO:tasks.ceph.mds.b:Started 2023-10-06T01:39:39.311 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T01:39:39.311 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T01:39:39.313 INFO:tasks.ceph.mds.d:Started 2023-10-06T01:39:39.314 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T01:39:39.314 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:39.390 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2023-10-06T01:39:39.390 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2023-10-06T01:39:39.395 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-06T01:39:39.400 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-06T01:39:39.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.446+0000 7f4691d9f700 1 Processor -- start 2023-10-06T01:39:39.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.446+0000 7f4691d9f700 1 -- start start 2023-10-06T01:39:39.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4691d9f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f468c15b240 0x7f468c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4691d9f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f468c15f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4691d9f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c15eb60 con 0x7f468c06ebb0 2023-10-06T01:39:39.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4691d9f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c1611f0 con 0x7f468c15b240 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4691d9f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c15df50 con 0x7f468c068d10 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4683fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f468c06ebb0 0x7f468c068600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33632/0 (socket says 172.21.15.98:33632) 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.447+0000 7f4683fff700 1 -- 172.21.15.98:0/938384750 learned_addr learned my addr 172.21.15.98:0/938384750 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f469159e700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f468c15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4690d9d700 1 --2- 172.21.15.98:0/938384750 >> v2:172.21.15.184:3300/0 conn(0x7f468c15b240 0x7f468c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260222085 0 0) 0x7f468c15eb60 con 0x7f468c06ebb0 2023-10-06T01:39:39.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4674003570 con 0x7f468c06ebb0 2023-10-06T01:39:39.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353495763 0 0) 0x7f4674003570 con 0x7f468c06ebb0 2023-10-06T01:39:39.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 msgr2=0x7f468c15f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f468c15f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 >> v2:172.21.15.184:3300/0 conn(0x7f468c15b240 msgr2=0x7f468c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 --2- 172.21.15.98:0/938384750 >> v2:172.21.15.184:3300/0 conn(0x7f468c15b240 0x7f468c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c15e230 con 0x7f468c06ebb0 2023-10-06T01:39:39.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f467c001470 con 0x7f468c06ebb0 2023-10-06T01:39:39.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 >> v1:172.21.15.98:6789/0 conn(0x7f468c06ebb0 legacy=0x7f468c068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4674004810 0x7f4674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f46740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c15df50 con 0x7f4674004810 2023-10-06T01:39:39.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467400ba60 con 0x7f4674003a60 2023-10-06T01:39:39.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467400bba0 con 0x7f468c068d10 2023-10-06T01:39:39.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4674004810 0x7f4674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4690d9d700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f469159e700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f46740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 -- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 msgr2=0x7f46740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c068d10 0x7f46740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 -- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674003a60 msgr2=0x7f4674003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4683fff700 1 -- 172.21.15.98:0/938384750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674003570 con 0x7f4674004810 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.448+0000 7f4690d9d700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674003a60 0x7f4674003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4683fff700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4674004810 0x7f4674004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c3c0 tx=0x7f467c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc98709f295fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00cc50 con 0x7f4674004810 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c00cdd0 con 0x7f4674004810 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f46837fe700 1 -- 172.21.15.98:0/938384750 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c005b70 con 0x7f4674004810 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 -- 172.21.15.98:0/938384750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4674004810 msgr2=0x7f4674004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 --2- 172.21.15.98:0/938384750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4674004810 0x7f4674004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c3c0 tx=0x7f467c001e90 comp rx=0 tx=0).stop 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f469159e700 1 -- 172.21.15.98:0/938384750 reap_dead start 2023-10-06T01:39:39.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 -- 172.21.15.98:0/938384750 shutdown_connections 2023-10-06T01:39:39.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 -- 172.21.15.98:0/938384750 >> 172.21.15.98:0/938384750 conn(0x7f468c0c9c50 msgr2=0x7f468c06db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:39.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 -- 172.21.15.98:0/938384750 shutdown_connections 2023-10-06T01:39:39.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.449+0000 7f4691d9f700 1 -- 172.21.15.98:0/938384750 wait complete. 2023-10-06T01:39:39.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.450+0000 7f4691d9f700 1 Processor -- start 2023-10-06T01:39:39.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 -- start start 2023-10-06T01:39:39.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674004810 0x7f468c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c15ac30 0x7f468c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0ad360 con 0x7f468c162d00 2023-10-06T01:39:39.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0af670 con 0x7f4674004810 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4691d9f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0b01b0 con 0x7f468c15ac30 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4683fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4683fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59306/0 (socket says 172.21.15.98:59306) 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4683fff700 1 -- 172.21.15.98:0/166254866 learned_addr learned my addr 172.21.15.98:0/166254866 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f4690d9d700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674004810 0x7f468c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.451+0000 7f469159e700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c15ac30 0x7f468c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 -- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c15ac30 msgr2=0x7f468c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c15ac30 0x7f468c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 -- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674004810 msgr2=0x7f468c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674004810 0x7f468c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 -- 172.21.15.98:0/166254866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0af880 con 0x7f468c162d00 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4683fff700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46740046d0 tx=0x7f467c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c001430 con 0x7f468c162d00 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0072e0 con 0x7f468c162d00 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c0afb60 con 0x7f468c162d00 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0076a0 con 0x7f468c162d00 2023-10-06T01:39:39.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.452+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c0ad050 con 0x7f468c162d00 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.453+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c05f270 con 0x7f468c162d00 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.453+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f467c019030 con 0x7f468c162d00 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.454+0000 7f4680ff9700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 0x7f4678069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.454+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f467c084ee0 con 0x7f468c162d00 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.454+0000 7f4690d9d700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 0x7f4678069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.454+0000 7f4690d9d700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 0x7f4678069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46880094d0 tx=0x7f468800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.458+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f467c052420 con 0x7f468c162d00 2023-10-06T01:39:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.659+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f468c15e510 con 0x7f468c162d00 2023-10-06T01:39:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.659+0000 7f4680ff9700 1 -- 172.21.15.98:0/166254866 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13770 (secure 0 0 0) 0x7f467c055a40 con 0x7f468c162d00 2023-10-06T01:39:39.661 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:39.661 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":15,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:39.280028+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:39.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 msgr2=0x7f4678069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 0x7f4678069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46880094d0 tx=0x7f468800b040 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 msgr2=0x7f468c15a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46740046d0 tx=0x7f467c015000 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 shutdown_connections 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4678066ea0 0x7f4678069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c15ac30 0x7f468c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4674004810 0x7f468c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 --2- 172.21.15.98:0/166254866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f468c162d00 0x7f468c15a6f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 >> 172.21.15.98:0/166254866 conn(0x7f468c0c9c50 msgr2=0x7f468c0ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:39.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 shutdown_connections 2023-10-06T01:39:39.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.662+0000 7f4691d9f700 1 -- 172.21.15.98:0/166254866 wait complete. 2023-10-06T01:39:39.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T01:39:39.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:39:39.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.797+0000 7f217c094700 1 Processor -- start 2023-10-06T01:39:39.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.797+0000 7f217c094700 1 -- start start 2023-10-06T01:39:39.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2174159510 0x7f21740cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21741598e0 0x7f21740cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21740ce820 0x7f21740cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21740cf610 con 0x7f2174159510 2023-10-06T01:39:39.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21740cf8f0 con 0x7f21740ce820 2023-10-06T01:39:39.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217c094700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740cfbd0 con 0x7f21741598e0 2023-10-06T01:39:39.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2174159510 0x7f21740cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2174159510 0x7f21740cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59318/0 (socket says 172.21.15.98:59318) 2023-10-06T01:39:39.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 -- 172.21.15.98:0/718912068 learned_addr learned my addr 172.21.15.98:0/718912068 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:39.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217962f700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21741598e0 0x7f21740cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f217a631700 1 --2- 172.21.15.98:0/718912068 >> v2:172.21.15.184:3300/0 conn(0x7f21740ce820 0x7f21740cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 -- 172.21.15.98:0/718912068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21741598e0 msgr2=0x7f21740cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21741598e0 0x7f21740cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 -- 172.21.15.98:0/718912068 >> v2:172.21.15.184:3300/0 conn(0x7f21740ce820 msgr2=0x7f21740cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:39.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 --2- 172.21.15.98:0/718912068 >> v2:172.21.15.184:3300/0 conn(0x7f21740ce820 0x7f21740cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 -- 172.21.15.98:0/718912068 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740cfeb0 con 0x7f2174159510 2023-10-06T01:39:39.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2179e30700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f21740cb5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f217408acb0 tx=0x7f2170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e721d53d1ff2939 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2178e2e700 1 -- 172.21.15.98:0/718912068 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000b860 con 0x7f2174159510 2023-10-06T01:39:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2178e2e700 1 -- 172.21.15.98:0/718912068 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000b9c0 con 0x7f2174159510 2023-10-06T01:39:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.798+0000 7f2178e2e700 1 -- 172.21.15.98:0/718912068 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000bd60 con 0x7f2174159510 2023-10-06T01:39:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 -- 172.21.15.98:0/718912068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 msgr2=0x7f21740cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f21740cb5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f217408acb0 tx=0x7f2170009250 comp rx=0 tx=0).stop 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 -- 172.21.15.98:0/718912068 shutdown_connections 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 --2- 172.21.15.98:0/718912068 >> v2:172.21.15.184:3300/0 conn(0x7f21740ce820 0x7f21740cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f21740cb5a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 --2- 172.21.15.98:0/718912068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21741598e0 0x7f21740cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 -- 172.21.15.98:0/718912068 >> 172.21.15.98:0/718912068 conn(0x7f2174089e90 msgr2=0x7f21740997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 -- 172.21.15.98:0/718912068 shutdown_connections 2023-10-06T01:39:39.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.799+0000 7f217c094700 1 -- 172.21.15.98:0/718912068 wait complete. 2023-10-06T01:39:39.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.800+0000 7f217c094700 1 Processor -- start 2023-10-06T01:39:39.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.800+0000 7f217c094700 1 -- start start 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21740ce820 0x7f217416e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21741598e0 0x7f2174069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217415bcc0 con 0x7f2174159510 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217416e5f0 con 0x7f21741598e0 2023-10-06T01:39:39.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217415c590 con 0x7f21740ce820 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59328/0 (socket says 172.21.15.98:59328) 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 -- 172.21.15.98:0/3188273321 learned_addr learned my addr 172.21.15.98:0/3188273321 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217a631700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21741598e0 0x7f2174069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f2179e30700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21740ce820 0x7f217416e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 -- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21740ce820 msgr2=0x7f217416e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21740ce820 0x7f217416e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 -- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21741598e0 msgr2=0x7f2174069aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21741598e0 0x7f2174069aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 -- 172.21.15.98:0/3188273321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700094f0 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217962f700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f216400d860 tx=0x7f2164008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640125f0 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164012750 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217415b0b0 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.801+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164012af0 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.802+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217415b670 con 0x7f2174159510 2023-10-06T01:39:39.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.802+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2174064ad0 con 0x7f2174159510 2023-10-06T01:39:39.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.803+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2164013480 con 0x7f2174159510 2023-10-06T01:39:39.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.803+0000 7f2169ffb700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 0x7f2158069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:39.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.803+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2164084580 con 0x7f2174159510 2023-10-06T01:39:39.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.807+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2164051ac0 con 0x7f2174159510 2023-10-06T01:39:39.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.813+0000 7f2179e30700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 0x7f2158069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:39.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.813+0000 7f2179e30700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 0x7f2158069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f217408acb0 tx=0x7f2170017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:39.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.993+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f21740cf180 con 0x7f2174159510 2023-10-06T01:39:39.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.993+0000 7f2169ffb700 1 -- 172.21.15.98:0/3188273321 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f21640550e0 con 0x7f2174159510 2023-10-06T01:39:39.995 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:39:39.997 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 msgr2=0x7f2158069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.997 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 0x7f2158069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f217408acb0 tx=0x7f2170017000 comp rx=0 tx=0).stop 2023-10-06T01:39:39.997 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 msgr2=0x7f2174069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:39.997 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f216400d860 tx=0x7f2164008040 comp rx=0 tx=0).stop 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 shutdown_connections 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2158066ea0 0x7f2158069350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21741598e0 0x7f2174069aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21740ce820 0x7f217416e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 --2- 172.21.15.98:0/3188273321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2174159510 0x7f2174069560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 >> 172.21.15.98:0/3188273321 conn(0x7f2174089e90 msgr2=0x7f217415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 shutdown_connections 2023-10-06T01:39:39.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:39.995+0000 7f217c094700 1 -- 172.21.15.98:0/3188273321 wait complete. 2023-10-06T01:39:40.009 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:39:40.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.128+0000 7fa6f0b4e700 1 Processor -- start 2023-10-06T01:39:40.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.128+0000 7fa6f0b4e700 1 -- start start 2023-10-06T01:39:40.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6f0b4e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6ec0cc860 0x7fa6ec0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6f0b4e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa6ec0675f0 0x7fa6ec067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6f0b4e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec067f70 con 0x7fa6ec06eff0 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6f0b4e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec159c00 con 0x7fa6ec0cc860 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6f0b4e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec06f460 con 0x7fa6ec0675f0 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6ea59c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6ec0cc860 0x7fa6ec0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6e9d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa6ec06eff0 0x7fa6ec0c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33644/0 (socket says 172.21.15.98:33644) 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6e9d9b700 1 -- 172.21.15.98:0/2495838490 learned_addr learned my addr 172.21.15.98:0/2495838490 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.129+0000 7fa6ead9d700 1 --2- 172.21.15.98:0/2495838490 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec0675f0 0x7fa6ec067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6ead9d700 1 -- 172.21.15.98:0/2495838490 >> v2:172.21.15.98:3300/0 conn(0x7fa6ec0cc860 msgr2=0x7fa6ec0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6ead9d700 1 --2- 172.21.15.98:0/2495838490 >> v2:172.21.15.98:3300/0 conn(0x7fa6ec0cc860 0x7fa6ec0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6ead9d700 1 -- 172.21.15.98:0/2495838490 >> v1:172.21.15.98:6789/0 conn(0x7fa6ec06eff0 legacy=0x7fa6ec0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6ead9d700 1 -- 172.21.15.98:0/2495838490 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0cbfb0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6ead9d700 1 --2- 172.21.15.98:0/2495838490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec067a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0cc1a0 tx=0x7fa6e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409c494826ae4faf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6e959a700 1 -- 172.21.15.98:0/2495838490 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4016070 con 0x7fa6ec0675f0 2023-10-06T01:39:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6e959a700 1 -- 172.21.15.98:0/2495838490 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400bb30 con 0x7fa6ec0675f0 2023-10-06T01:39:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6e959a700 1 -- 172.21.15.98:0/2495838490 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400e070 con 0x7fa6ec0675f0 2023-10-06T01:39:40.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/2495838490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec0675f0 msgr2=0x7fa6ec067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.130+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/2495838490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec067a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0cc1a0 tx=0x7fa6e4009780 comp rx=0 tx=0).stop 2023-10-06T01:39:40.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/2495838490 shutdown_connections 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/2495838490 >> v2:172.21.15.98:3300/0 conn(0x7fa6ec0cc860 0x7fa6ec0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/2495838490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec067a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/2495838490 >> 172.21.15.98:0/2495838490 conn(0x7fa6ec0691d0 msgr2=0x7fa6ec0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/2495838490 shutdown_connections 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.131+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/2495838490 wait complete. 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 Processor -- start 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 -- start start 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 0x7fa6ec146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 0x7fa6ec15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1581d0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec147150 con 0x7fa6ec06eff0 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6f0b4e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15bf50 con 0x7fa6ec0cc860 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6ea59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6ea59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59362/0 (socket says 172.21.15.98:59362) 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.132+0000 7fa6ea59c700 1 -- 172.21.15.98:0/1215648101 learned_addr learned my addr 172.21.15.98:0/1215648101 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ead9d700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 0x7fa6ec15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6e9d9b700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 0x7fa6ec146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 -- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 msgr2=0x7fa6ec15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 0x7fa6ec15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 -- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 msgr2=0x7fa6ec146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 0x7fa6ec146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 -- 172.21.15.98:0/1215648101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009a20 con 0x7fa6ec0675f0 2023-10-06T01:39:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ead9d700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 0x7fa6ec15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6e9d9b700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 0x7fa6ec146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.133+0000 7fa6ea59c700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e930 tx=0x7fa6dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.152+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc009510 con 0x7fa6ec0675f0 2023-10-06T01:39:40.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.152+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec15c230 con 0x7fa6ec0675f0 2023-10-06T01:39:40.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.152+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec157e60 con 0x7fa6ec0675f0 2023-10-06T01:39:40.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.152+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc00ca60 con 0x7fa6ec0675f0 2023-10-06T01:39:40.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.153+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc00cbd0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.154+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d00052b0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.154+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa6dc013470 con 0x7fa6ec0675f0 2023-10-06T01:39:40.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.155+0000 7fa6da7fc700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 0x7fa6cc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.155+0000 7fa6e9d9b700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 0x7fa6cc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.155+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa6dc0846d0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.155+0000 7fa6e9d9b700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 0x7fa6cc069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec15d5b0 tx=0x7fa6e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.159+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6dc051c10 con 0x7fa6ec0675f0 2023-10-06T01:39:40.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.347+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6d0005ae0 con 0x7fa6ec0675f0 2023-10-06T01:39:40.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.347+0000 7fa6da7fc700 1 -- 172.21.15.98:0/1215648101 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6dc055230 con 0x7fa6ec0675f0 2023-10-06T01:39:40.349 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:39:40.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 msgr2=0x7fa6cc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 0x7fa6cc069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec15d5b0 tx=0x7fa6e0009040 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 msgr2=0x7fa6ec156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e930 tx=0x7fa6dc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 shutdown_connections 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6cc066ea0 0x7fa6cc069350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cc860 0x7fa6ec15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ec0675f0 0x7fa6ec156390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 --2- 172.21.15.98:0/1215648101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06eff0 0x7fa6ec146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 >> 172.21.15.98:0/1215648101 conn(0x7fa6ec0691d0 msgr2=0x7fa6ec063580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 shutdown_connections 2023-10-06T01:39:40.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.350+0000 7fa6f0b4e700 1 -- 172.21.15.98:0/1215648101 wait complete. 2023-10-06T01:39:40.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:39:40.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.480+0000 7f6e38e24700 1 Processor -- start 2023-10-06T01:39:40.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.480+0000 7f6e38e24700 1 -- start start 2023-10-06T01:39:40.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e38e24700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e341595a0 0x7f6e340aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e38e24700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e340aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e38e24700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e340b0ca0 con 0x7f6e340b07a0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e38e24700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340b0f80 con 0x7f6e341595a0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e38e24700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b1260 con 0x7f6e340b49e0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e32d9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6e340b07a0 0x7f6e340b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33660/0 (socket says 172.21.15.98:33660) 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e32d9d700 1 -- 172.21.15.98:0/3355192903 learned_addr learned my addr 172.21.15.98:0/3355192903 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3259c700 1 --2- 172.21.15.98:0/3355192903 >> v2:172.21.15.184:3300/0 conn(0x7f6e341595a0 0x7f6e340aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e340aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621017650 0 0) 0x7f6e340b0ca0 con 0x7f6e340b07a0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e1c003570 con 0x7f6e340b07a0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195179371 0 0) 0x7f6e1c003570 con 0x7f6e340b07a0 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 msgr2=0x7f6e340aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e340aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 >> v2:172.21.15.184:3300/0 conn(0x7f6e341595a0 msgr2=0x7f6e340aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 --2- 172.21.15.98:0/3355192903 >> v2:172.21.15.184:3300/0 conn(0x7f6e341595a0 0x7f6e340aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340b1540 con 0x7f6e340b07a0 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f6e2c001470 con 0x7f6e340b07a0 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 >> v1:172.21.15.98:6789/0 conn(0x7f6e340b07a0 legacy=0x7f6e340b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e1c004810 0x7f6e1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b0f80 con 0x7f6e1c003a60 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b1260 con 0x7f6e1c004810 2023-10-06T01:39:40.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c00ba60 con 0x7f6e340b49e0 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.481+0000 7f6e3259c700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e1c004810 0x7f6e1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e32d9d700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 -- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 msgr2=0x7f6e1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e340b49e0 0x7f6e1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 -- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e1c004810 msgr2=0x7f6e1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e1c004810 0x7f6e1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 -- 172.21.15.98:0/3355192903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c003570 con 0x7f6e1c003a60 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3259c700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e1c003e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e300 tx=0x7f6e2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f6e2df31ac955 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400eb50 con 0x7f6e1c003a60 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2400f040 con 0x7f6e1c003a60 2023-10-06T01:39:40.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e3159a700 1 -- 172.21.15.98:0/3355192903 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24014480 con 0x7f6e1c003a60 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e38e24700 1 -- 172.21.15.98:0/3355192903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 msgr2=0x7f6e1c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e38e24700 1 --2- 172.21.15.98:0/3355192903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e1c003e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e300 tx=0x7f6e2400b040 comp rx=0 tx=0).stop 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.482+0000 7f6e32d9d700 1 -- 172.21.15.98:0/3355192903 reap_dead start 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.483+0000 7f6e38e24700 1 -- 172.21.15.98:0/3355192903 shutdown_connections 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.483+0000 7f6e38e24700 1 -- 172.21.15.98:0/3355192903 >> 172.21.15.98:0/3355192903 conn(0x7f6e34099830 msgr2=0x7f6e3406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.483+0000 7f6e38e24700 1 -- 172.21.15.98:0/3355192903 shutdown_connections 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.483+0000 7f6e38e24700 1 -- 172.21.15.98:0/3355192903 wait complete. 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 Processor -- start 2023-10-06T01:39:40.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 -- start start 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 0x7f6e34151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 0x7f6e34153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415e580 con 0x7f6e1c003a60 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415d970 con 0x7f6e34153390 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e38e24700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415dc50 con 0x7f6e3415adc0 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e3259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e3259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59376/0 (socket says 172.21.15.98:59376) 2023-10-06T01:39:40.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.484+0000 7f6e3259c700 1 -- 172.21.15.98:0/351594684 learned_addr learned my addr 172.21.15.98:0/351594684 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 0x7f6e34151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e32d9d700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 0x7f6e34153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 -- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 msgr2=0x7f6e34151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 0x7f6e34151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 -- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 msgr2=0x7f6e34153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 0x7f6e34153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 -- 172.21.15.98:0/351594684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415df30 con 0x7f6e1c003a60 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e32d9d700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 0x7f6e34153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 0x7f6e34151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e3259c700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0046d0 tx=0x7f6e2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24014e80 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2401d630 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3415e210 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401da10 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.485+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3415b230 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.486+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34064ad0 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.486+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6e24003040 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.487+0000 7f6e227fc700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 0x7f6e14069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.487+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6e24086260 con 0x7f6e1c003a60 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.487+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 0x7f6e14069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.487+0000 7f6e31d9b700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 0x7f6e14069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a1a0 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.491+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e240537a0 con 0x7f6e1c003a60 2023-10-06T01:39:40.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.523+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e24053b70 con 0x7f6e1c003a60 2023-10-06T01:39:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.717+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e340b1820 con 0x7f6e1c003a60 2023-10-06T01:39:40.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.717+0000 7f6e227fc700 1 -- 172.21.15.98:0/351594684 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f6e24018de0 con 0x7f6e1c003a60 2023-10-06T01:39:40.719 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:40.719 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T01:39:40.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 msgr2=0x7f6e14069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 0x7f6e14069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a1a0 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 msgr2=0x7f6e3415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0046d0 tx=0x7f6e2401a000 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 shutdown_connections 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6e14066ea0 0x7f6e14069350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e34153390 0x7f6e34153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e1c003a60 0x7f6e3415d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 --2- 172.21.15.98:0/351594684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3415adc0 0x7f6e34151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 >> 172.21.15.98:0/351594684 conn(0x7f6e34099830 msgr2=0x7f6e340cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 shutdown_connections 2023-10-06T01:39:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.720+0000 7f6e38e24700 1 -- 172.21.15.98:0/351594684 wait complete. 2023-10-06T01:39:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 3 2023-10-06T01:39:40.733 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:40.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.845+0000 7f95d30c5700 1 Processor -- start 2023-10-06T01:39:40.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.845+0000 7f95d30c5700 1 -- start start 2023-10-06T01:39:40.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d30c5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95cc157460 0x7f95cc0ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d30c5700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f95cc0ac480 0x7f95cc068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d30c5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0cae60 con 0x7f95cc157830 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d30c5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0aeed0 con 0x7f95cc157460 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d30c5700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0af1b0 con 0x7f95cc0ac480 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d0e61700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95cc157460 0x7f95cc0ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d0e61700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95cc157460 0x7f95cc0ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59390/0 (socket says 172.21.15.98:59390) 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d0e61700 1 -- 172.21.15.98:0/1252403565 learned_addr learned my addr 172.21.15.98:0/1252403565 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.846+0000 7f95d1662700 1 --2- 172.21.15.98:0/1252403565 >> v2:172.21.15.184:3300/0 conn(0x7f95cc0ac480 0x7f95cc068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d0e61700 1 -- 172.21.15.98:0/1252403565 >> v2:172.21.15.184:3300/0 conn(0x7f95cc0ac480 msgr2=0x7f95cc068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d0e61700 1 --2- 172.21.15.98:0/1252403565 >> v2:172.21.15.184:3300/0 conn(0x7f95cc0ac480 0x7f95cc068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d0e61700 1 -- 172.21.15.98:0/1252403565 >> v1:172.21.15.184:6789/0 conn(0x7f95cc157830 legacy=0x7f95cc0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d0e61700 1 -- 172.21.15.98:0/1252403565 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0af490 con 0x7f95cc157460 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d0e61700 1 --2- 172.21.15.98:0/1252403565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc157460 0x7f95cc0ae8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c9660 tx=0x7f95c8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128b7d25bd5cf3a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95c37fe700 1 -- 172.21.15.98:0/1252403565 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8005c10 con 0x7f95cc157460 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95c37fe700 1 -- 172.21.15.98:0/1252403565 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8005d70 con 0x7f95cc157460 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95c37fe700 1 -- 172.21.15.98:0/1252403565 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8006260 con 0x7f95cc157460 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 -- 172.21.15.98:0/1252403565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc157460 msgr2=0x7f95cc0ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 --2- 172.21.15.98:0/1252403565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc157460 0x7f95cc0ae8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c9660 tx=0x7f95c8009220 comp rx=0 tx=0).stop 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 -- 172.21.15.98:0/1252403565 shutdown_connections 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 --2- 172.21.15.98:0/1252403565 >> v2:172.21.15.184:3300/0 conn(0x7f95cc0ac480 0x7f95cc068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 --2- 172.21.15.98:0/1252403565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc157460 0x7f95cc0ae8f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.847+0000 7f95d30c5700 1 -- 172.21.15.98:0/1252403565 >> 172.21.15.98:0/1252403565 conn(0x7f95cc06de80 msgr2=0x7f95cc06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:40.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.848+0000 7f95d30c5700 1 -- 172.21.15.98:0/1252403565 shutdown_connections 2023-10-06T01:39:40.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.848+0000 7f95d30c5700 1 -- 172.21.15.98:0/1252403565 wait complete. 2023-10-06T01:39:40.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 Processor -- start 2023-10-06T01:39:40.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 -- start start 2023-10-06T01:39:40.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 0x7f95cc0aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 0x7f95cc0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0c28c0 con 0x7f95cc0ac480 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.849+0000 7f95d30c5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0c2ba0 con 0x7f95cc157830 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d30c5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0c2e80 con 0x7f95cc157460 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59400/0 (socket says 172.21.15.98:59400) 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 -- 172.21.15.98:0/4162627962 learned_addr learned my addr 172.21.15.98:0/4162627962 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d1662700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 0x7f95cc0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95c3fff700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 0x7f95cc0aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 -- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 msgr2=0x7f95cc0aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 0x7f95cc0aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 -- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 msgr2=0x7f95cc0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 0x7f95cc0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 -- 172.21.15.98:0/4162627962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009480 con 0x7f95cc0ac480 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d1662700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 0x7f95cc0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95c3fff700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 0x7f95cc0aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d0e61700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b0e0 tx=0x7f95c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800b900 con 0x7f95cc0ac480 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8020070 con 0x7f95cc0ac480 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800bc90 con 0x7f95cc0ac480 2023-10-06T01:39:40.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc0c3160 con 0x7f95cc0ac480 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.850+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc148550 con 0x7f95cc0ac480 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.851+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc0c3680 con 0x7f95cc0ac480 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.852+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95c8022030 con 0x7f95cc0ac480 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.852+0000 7f95c0ff9700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 0x7f95b8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.852+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f95c8085ef0 con 0x7f95cc0ac480 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.852+0000 7f95c3fff700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 0x7f95b8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.852+0000 7f95c3fff700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 0x7f95b8069410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c5da0 tx=0x7f95bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:40.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:40.856+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95cc0c3680 con 0x7f95cc0ac480 2023-10-06T01:39:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.042+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95cc0c3680 con 0x7f95cc0ac480 2023-10-06T01:39:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.043+0000 7f95c0ff9700 1 -- 172.21.15.98:0/4162627962 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13744 (secure 0 0 0) 0x7f95cc0c3680 con 0x7f95cc0ac480 2023-10-06T01:39:41.045 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:41.045 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":16,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:40.283616+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:41.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 msgr2=0x7f95b8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 0x7f95b8069410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c5da0 tx=0x7f95bc009040 comp rx=0 tx=0).stop 2023-10-06T01:39:41.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 msgr2=0x7f95cc148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b0e0 tx=0x7f95c8016000 comp rx=0 tx=0).stop 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 shutdown_connections 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95b8066f60 0x7f95b8069410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95cc157830 0x7f95cc0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95cc0ac480 0x7f95cc148010 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 --2- 172.21.15.98:0/4162627962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95cc157460 0x7f95cc0aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.045+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 >> 172.21.15.98:0/4162627962 conn(0x7f95cc06de80 msgr2=0x7f95cc068830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.046+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 shutdown_connections 2023-10-06T01:39:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.046+0000 7f95d30c5700 1 -- 172.21.15.98:0/4162627962 wait complete. 2023-10-06T01:39:41.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T01:39:41.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:39:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.174+0000 7efd40a7a700 1 Processor -- start 2023-10-06T01:39:41.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.174+0000 7efd40a7a700 1 -- start start 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd40a7a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd3c1598e0 0x7efd3c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd40a7a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c0cbcb0 0x7efd3c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd40a7a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c0cf610 con 0x7efd3c159510 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd40a7a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cf8f0 con 0x7efd3c1598e0 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd40a7a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0cfbd0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd3c1598e0 0x7efd3c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd3c1598e0 0x7efd3c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59406/0 (socket says 172.21.15.98:59406) 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 -- 172.21.15.98:0/3874077121 learned_addr learned my addr 172.21.15.98:0/3874077121 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd3ad9d700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c0cbcb0 0x7efd3c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 -- 172.21.15.98:0/3874077121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c0cbcb0 msgr2=0x7efd3c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c0cbcb0 0x7efd3c0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 -- 172.21.15.98:0/3874077121 >> v1:172.21.15.184:6789/0 conn(0x7efd3c159510 legacy=0x7efd3c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 -- 172.21.15.98:0/3874077121 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0cfeb0 con 0x7efd3c1598e0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.175+0000 7efd39d9b700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c1598e0 0x7efd3c0ce820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd300099b0 tx=0x7efd30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12432701245f1dac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd3959a700 1 -- 172.21.15.98:0/3874077121 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30009ea0 con 0x7efd3c1598e0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd3959a700 1 -- 172.21.15.98:0/3874077121 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3000b860 con 0x7efd3c1598e0 2023-10-06T01:39:41.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd3959a700 1 -- 172.21.15.98:0/3874077121 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000bc00 con 0x7efd3c1598e0 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 -- 172.21.15.98:0/3874077121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c1598e0 msgr2=0x7efd3c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c1598e0 0x7efd3c0ce820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd300099b0 tx=0x7efd30009250 comp rx=0 tx=0).stop 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 -- 172.21.15.98:0/3874077121 shutdown_connections 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c0cbcb0 0x7efd3c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 --2- 172.21.15.98:0/3874077121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c1598e0 0x7efd3c0ce820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 -- 172.21.15.98:0/3874077121 >> 172.21.15.98:0/3874077121 conn(0x7efd3c089e90 msgr2=0x7efd3c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 -- 172.21.15.98:0/3874077121 shutdown_connections 2023-10-06T01:39:41.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.176+0000 7efd40a7a700 1 -- 172.21.15.98:0/3874077121 wait complete. 2023-10-06T01:39:41.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.177+0000 7efd40a7a700 1 Processor -- start 2023-10-06T01:39:41.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.177+0000 7efd40a7a700 1 -- start start 2023-10-06T01:39:41.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c159510 0x7efd3c16dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3c1598e0 0x7efd3c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c16a120 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c155650 con 0x7efd3c1598e0 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd40a7a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c155930 con 0x7efd3c159510 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59422/0 (socket says 172.21.15.98:59422) 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 -- 172.21.15.98:0/2478943164 learned_addr learned my addr 172.21.15.98:0/2478943164 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3ad9d700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3c1598e0 0x7efd3c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd39d9b700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c159510 0x7efd3c16dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 -- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c159510 msgr2=0x7efd3c16dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c159510 0x7efd3c16dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 -- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3c1598e0 msgr2=0x7efd3c149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3c1598e0 0x7efd3c149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.178+0000 7efd3a59c700 1 -- 172.21.15.98:0/2478943164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300094f0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd3a59c700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c790 tx=0x7efd34007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011590 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34011710 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c155c10 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011ad0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.179+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c16a5b0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.180+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c064ad0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.180+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd3401b3b0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.180+0000 7efd2a7fc700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 0x7efd1c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.180+0000 7efd39d9b700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 0x7efd1c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.180+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efd340837a0 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.181+0000 7efd39d9b700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 0x7efd1c069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd3c08a870 tx=0x7efd30017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.185+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd34050b60 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.366+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efd3c16a400 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.367+0000 7efd2a7fc700 1 -- 172.21.15.98:0/2478943164 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd34017020 con 0x7efd3c0cbcb0 2023-10-06T01:39:41.368 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:39:41.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 msgr2=0x7efd1c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 0x7efd1c069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd3c08a870 tx=0x7efd30017000 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 msgr2=0x7efd3c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c790 tx=0x7efd34007f40 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 shutdown_connections 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd1c066fb0 0x7efd1c069460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3c1598e0 0x7efd3c149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd3c0cbcb0 0x7efd3c155fe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 --2- 172.21.15.98:0/2478943164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd3c159510 0x7efd3c16dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 >> 172.21.15.98:0/2478943164 conn(0x7efd3c089e90 msgr2=0x7efd3c14b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 shutdown_connections 2023-10-06T01:39:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.369+0000 7efd40a7a700 1 -- 172.21.15.98:0/2478943164 wait complete. 2023-10-06T01:39:41.382 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:39:41.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.498+0000 7f7cdffff700 1 Processor -- start 2023-10-06T01:39:41.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.498+0000 7f7cdffff700 1 -- start start 2023-10-06T01:39:41.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00b0460 0x7f7ce00cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ce00c4180 0x7f7ce00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00cdd00 con 0x7f7ce00b0830 2023-10-06T01:39:41.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdffff700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00cd0f0 con 0x7f7ce00c4180 2023-10-06T01:39:41.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00cd3d0 con 0x7f7ce00b0460 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ce00c4180 0x7f7ce00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ce00c4180 0x7f7ce00ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59428/0 (socket says 172.21.15.98:59428) 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 -- 172.21.15.98:0/3456718485 learned_addr learned my addr 172.21.15.98:0/3456718485 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdeffd700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00b0460 0x7f7ce00cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 -- 172.21.15.98:0/3456718485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00b0460 msgr2=0x7f7ce00cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00b0460 0x7f7ce00cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 -- 172.21.15.98:0/3456718485 >> v1:172.21.15.184:6789/0 conn(0x7f7ce00b0830 legacy=0x7f7ce00c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.499+0000 7f7cdf7fe700 1 -- 172.21.15.98:0/3456718485 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00cd6b0 con 0x7f7ce00c4180 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdf7fe700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00c4180 0x7f7ce00ccae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009f50 tx=0x7f7cd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fa168607d3ce3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cde7fc700 1 -- 172.21.15.98:0/3456718485 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800e070 con 0x7f7ce00c4180 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cde7fc700 1 -- 172.21.15.98:0/3456718485 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd800b9c0 con 0x7f7ce00c4180 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cde7fc700 1 -- 172.21.15.98:0/3456718485 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80092e0 con 0x7f7ce00c4180 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 -- 172.21.15.98:0/3456718485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00c4180 msgr2=0x7f7ce00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00c4180 0x7f7ce00ccae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009f50 tx=0x7f7cd8009840 comp rx=0 tx=0).stop 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 -- 172.21.15.98:0/3456718485 shutdown_connections 2023-10-06T01:39:41.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00c4180 0x7f7ce00ccae0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 --2- 172.21.15.98:0/3456718485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00b0460 0x7f7ce00cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 -- 172.21.15.98:0/3456718485 >> 172.21.15.98:0/3456718485 conn(0x7f7ce0096a70 msgr2=0x7f7ce0096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 -- 172.21.15.98:0/3456718485 shutdown_connections 2023-10-06T01:39:41.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.500+0000 7f7cdffff700 1 -- 172.21.15.98:0/3456718485 wait complete. 2023-10-06T01:39:41.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.501+0000 7f7cdffff700 1 Processor -- start 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.501+0000 7f7cdffff700 1 -- start start 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce00b0830 0x7f7ce00c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00c4180 0x7f7ce0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce016dec0 con 0x7f7ce00b0460 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b2e70 con 0x7f7ce00b0830 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c7c50 con 0x7f7ce00c4180 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59430/0 (socket says 172.21.15.98:59430) 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdeffd700 1 -- 172.21.15.98:0/562366489 learned_addr learned my addr 172.21.15.98:0/562366489 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:41.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cdf7fe700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00c4180 0x7f7ce0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.502+0000 7f7cd67fc700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce00b0830 0x7f7ce00c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 -- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00c4180 msgr2=0x7f7ce0069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00c4180 0x7f7ce0069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 -- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce00b0830 msgr2=0x7f7ce00c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce00b0830 0x7f7ce00c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 -- 172.21.15.98:0/562366489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8009ae0 con 0x7f7ce00b0460 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdeffd700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e930 tx=0x7f7cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0013680 con 0x7f7ce00b0460 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd00137e0 con 0x7f7ce00b0460 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0013b80 con 0x7f7ce00b0460 2023-10-06T01:39:41.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00c7f30 con 0x7f7ce00b0460 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.503+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce00b2bb0 con 0x7f7ce00b0460 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.504+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce005f270 con 0x7f7ce00b0460 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.504+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cd000c3e0 con 0x7f7ce00b0460 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.505+0000 7f7cd77fe700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.505+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7cd00843f0 con 0x7f7ce00b0460 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.505+0000 7f7cd67fc700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.505+0000 7f7cd67fc700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 0x7f7cc00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007fd0 tx=0x7f7cc8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.509+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cd0051850 con 0x7f7ce00b0460 2023-10-06T01:39:41.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.690+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ce00cd990 con 0x7f7ce00b0460 2023-10-06T01:39:41.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.690+0000 7f7cd77fe700 1 -- 172.21.15.98:0/562366489 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cd0054e70 con 0x7f7ce00b0460 2023-10-06T01:39:41.692 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 msgr2=0x7f7cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 0x7f7cc00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007fd0 tx=0x7f7cc8007eb0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 msgr2=0x7f7ce016d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e930 tx=0x7f7cd00091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 shutdown_connections 2023-10-06T01:39:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce00c4180 0x7f7ce0069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ce00b0460 0x7f7ce016d980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 --2- 172.21.15.98:0/562366489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce00b0830 0x7f7ce00c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 >> 172.21.15.98:0/562366489 conn(0x7f7ce0096a70 msgr2=0x7f7ce00c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 shutdown_connections 2023-10-06T01:39:41.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.692+0000 7f7cdffff700 1 -- 172.21.15.98:0/562366489 wait complete. 2023-10-06T01:39:41.705 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T01:39:41.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.818+0000 7f4fc8639700 1 Processor -- start 2023-10-06T01:39:41.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.818+0000 7f4fc8639700 1 -- start start 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc8639700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fc0159f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc8639700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0068d50 con 0x7f4fc00cb630 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc8639700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc015a6b0 con 0x7f4fc0159b10 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc8639700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015a4d0 con 0x7f4fc00cadc0 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc63d5700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4fc00cb630 0x7f4fc00691e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33662/0 (socket says 172.21.15.98:33662) 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.819+0000 7f4fc63d5700 1 -- 172.21.15.98:0/4005491817 learned_addr learned my addr 172.21.15.98:0/4005491817 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fc0159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000836713 0 0) 0x7f4fc0068d50 con 0x7f4fc00cb630 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fac003570 con 0x7f4fc00cb630 2023-10-06T01:39:41.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532034127 0 0) 0x7f4fc015a6b0 con 0x7f4fc0159b10 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc0068d50 con 0x7f4fc0159b10 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2369262707 0 0) 0x7f4fac003570 con 0x7f4fc00cb630 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 msgr2=0x7f4fc0159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fc0159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 >> v1:172.21.15.184:6789/0 conn(0x7f4fc0159b10 legacy=0x7f4fc00ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc015a990 con 0x7f4fc00cb630 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f4fbc0017f0 con 0x7f4fc00cb630 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 >> v1:172.21.15.98:6789/0 conn(0x7f4fc00cb630 legacy=0x7f4fc00691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac003c20 0x7f4fac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015a4d0 con 0x7f4fac004d50 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac00bb40 con 0x7f4fac003c20 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac00bd00 con 0x7f4fc00cadc0 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc63d5700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac003c20 0x7f4fac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc5bd4700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 -- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 msgr2=0x7f4fac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00cadc0 0x7f4fac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 -- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac003c20 msgr2=0x7f4fac004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fac003c20 0x7f4fac004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.820+0000 7f4fc6bd6700 1 -- 172.21.15.98:0/4005491817 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac003570 con 0x7f4fac004d50 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fac0051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009c00 tx=0x7f4fb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9eb9e4fd029f57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800e710 con 0x7f4fac004d50 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800e870 con 0x7f4fac004d50 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc53d3700 1 -- 172.21.15.98:0/4005491817 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800ec30 con 0x7f4fac004d50 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 -- 172.21.15.98:0/4005491817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 msgr2=0x7f4fac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 --2- 172.21.15.98:0/4005491817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fac0051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009c00 tx=0x7f4fb80091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc6bd6700 1 -- 172.21.15.98:0/4005491817 reap_dead start 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 -- 172.21.15.98:0/4005491817 shutdown_connections 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 -- 172.21.15.98:0/4005491817 >> 172.21.15.98:0/4005491817 conn(0x7f4fc0070810 msgr2=0x7f4fc0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 -- 172.21.15.98:0/4005491817 shutdown_connections 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.821+0000 7f4fc8639700 1 -- 172.21.15.98:0/4005491817 wait complete. 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.822+0000 7f4fc8639700 1 Processor -- start 2023-10-06T01:39:41.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.822+0000 7f4fc8639700 1 -- start start 2023-10-06T01:39:41.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00c4440 0x7f4fc00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fc00c8460 0x7f4fc0153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00c7ff0 con 0x7f4fac004d50 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00c6a10 con 0x7f4fc00c8460 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc8639700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00c6cf0 con 0x7f4fc00c4440 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59440/0 (socket says 172.21.15.98:59440) 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 -- 172.21.15.98:0/2623877905 learned_addr learned my addr 172.21.15.98:0/2623877905 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc5bd4700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00c4440 0x7f4fc00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc6bd6700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fc00c8460 0x7f4fc0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 -- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00c4440 msgr2=0x7f4fc00c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00c4440 0x7f4fc00c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 -- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fc00c8460 msgr2=0x7f4fc0153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fc00c8460 0x7f4fc0153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.823+0000 7f4fc63d5700 1 -- 172.21.15.98:0/2623877905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00c6fd0 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fc63d5700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc007a90 tx=0x7f4fbc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc007ac0 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc00acc0 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00c72b0 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc013640 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.824+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00c6710 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.825+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc00c77d0 con 0x7f4fac004d50 2023-10-06T01:39:41.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.825+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fbc013a30 con 0x7f4fac004d50 2023-10-06T01:39:41.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.825+0000 7f4fb67fc700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:41.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.825+0000 7f4fc5bd4700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:41.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.825+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4fbc084f30 con 0x7f4fac004d50 2023-10-06T01:39:41.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.826+0000 7f4fc5bd4700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 0x7f4fa40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0168dc0 tx=0x7f4fb0016040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:41.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:41.829+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fc00c77d0 con 0x7f4fac004d50 2023-10-06T01:39:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.016+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4fc00c77d0 con 0x7f4fac004d50 2023-10-06T01:39:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.016+0000 7f4fb67fc700 1 -- 172.21.15.98:0/2623877905 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4fc00c77d0 con 0x7f4fac004d50 2023-10-06T01:39:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:42.018 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2023-10-06T01:39:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": "rbd" 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 2, 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:39:42.019 INFO:teuthology.orchestra.run.smithi098.stdout:] 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 msgr2=0x7f4fa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 0x7f4fa40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0168dc0 tx=0x7f4fb0016040 comp rx=0 tx=0).stop 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 msgr2=0x7f4fc00c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc007a90 tx=0x7f4fbc002190 comp rx=0 tx=0).stop 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 shutdown_connections 2023-10-06T01:39:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fc00c8460 0x7f4fc0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fac004d50 0x7f4fc00c7ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 --2- 172.21.15.98:0/2623877905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fc00c4440 0x7f4fc00c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 >> 172.21.15.98:0/2623877905 conn(0x7f4fc0070810 msgr2=0x7f4fc00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.018+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 shutdown_connections 2023-10-06T01:39:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.019+0000 7f4fc8639700 1 -- 172.21.15.98:0/2623877905 wait complete. 2023-10-06T01:39:42.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T01:39:42.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T01:39:42.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.147+0000 7f279fa82700 1 Processor -- start 2023-10-06T01:39:42.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.148+0000 7f279fa82700 1 -- start start 2023-10-06T01:39:42.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279fa82700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f279815cf50 0x7f27980ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279fa82700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 0x7f27980c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279fa82700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980c8970 con 0x7f279815cb80 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279fa82700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980c8f30 con 0x7f279815cf50 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279fa82700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980cf290 con 0x7f27980cceb0 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f279815cf50 0x7f27980ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f279815cf50 0x7f27980ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59450/0 (socket says 172.21.15.98:59450) 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 -- 172.21.15.98:0/1336748704 learned_addr learned my addr 172.21.15.98:0/1336748704 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279e01f700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 0x7f27980c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 -- 172.21.15.98:0/1336748704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 msgr2=0x7f27980c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:42.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 0x7f27980c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 -- 172.21.15.98:0/1336748704 >> v1:172.21.15.184:6789/0 conn(0x7f279815cb80 legacy=0x7f27980cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 -- 172.21.15.98:0/1336748704 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980c42a0 con 0x7f279815cf50 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279e01f700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 0x7f27980c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279d01d700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cf50 0x7f27980ced50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27940099b0 tx=0x7f2794009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddb767d8f1431b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279c81c700 1 -- 172.21.15.98:0/1336748704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400b860 con 0x7f279815cf50 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279c81c700 1 -- 172.21.15.98:0/1336748704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279400b9c0 con 0x7f279815cf50 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.149+0000 7f279c81c700 1 -- 172.21.15.98:0/1336748704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400bd60 con 0x7f279815cf50 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 -- 172.21.15.98:0/1336748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cf50 msgr2=0x7f27980ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cf50 0x7f27980ced50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27940099b0 tx=0x7f2794009250 comp rx=0 tx=0).stop 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 -- 172.21.15.98:0/1336748704 shutdown_connections 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980cceb0 0x7f27980c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 --2- 172.21.15.98:0/1336748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cf50 0x7f27980ced50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 -- 172.21.15.98:0/1336748704 >> 172.21.15.98:0/1336748704 conn(0x7f2798088000 msgr2=0x7f27980ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:42.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 -- 172.21.15.98:0/1336748704 shutdown_connections 2023-10-06T01:39:42.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.150+0000 7f279fa82700 1 -- 172.21.15.98:0/1336748704 wait complete. 2023-10-06T01:39:42.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.151+0000 7f279fa82700 1 Processor -- start 2023-10-06T01:39:42.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.151+0000 7f279fa82700 1 -- start start 2023-10-06T01:39:42.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27980cceb0 0x7f279815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 0x7f279815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c7060 con 0x7f279815cb80 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c6450 con 0x7f27980cceb0 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c6730 con 0x7f279815cf50 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59454/0 (socket says 172.21.15.98:59454) 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 -- 172.21.15.98:0/1291150048 learned_addr learned my addr 172.21.15.98:0/1291150048 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d81e700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27980cceb0 0x7f279815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279e01f700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 0x7f279815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 -- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 msgr2=0x7f279815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 0x7f279815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 -- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27980cceb0 msgr2=0x7f279815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27980cceb0 0x7f279815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 -- 172.21.15.98:0/1291150048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940094f0 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279e01f700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 0x7f279815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279d01d700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2794009290 tx=0x7f2794017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794026480 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279401f070 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980c6a10 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.152+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794026830 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.153+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279815e160 con 0x7f279815cb80 2023-10-06T01:39:42.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.153+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27980c71d0 con 0x7f279815cb80 2023-10-06T01:39:42.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.154+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f279401a430 con 0x7f279815cb80 2023-10-06T01:39:42.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.154+0000 7f2789ffb700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 0x7f277c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:42.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.154+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2794021070 con 0x7f279815cb80 2023-10-06T01:39:42.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.154+0000 7f279d81e700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 0x7f277c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:42.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.154+0000 7f279d81e700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 0x7f277c0693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f278c007fd0 tx=0x7f278c007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:42.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.158+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27940b75a0 con 0x7f279815cb80 2023-10-06T01:39:42.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:42.343+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2798064ad0 con 0x7f279815cb80 2023-10-06T01:39:43.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.302+0000 7f2789ffb700 1 -- 172.21.15.98:0/1291150048 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f279402b0b0 con 0x7f279815cb80 2023-10-06T01:39:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 msgr2=0x7f277c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 0x7f277c0693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f278c007fd0 tx=0x7f278c007eb0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 msgr2=0x7f27980c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2794009290 tx=0x7f2794017000 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 shutdown_connections 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f277c066f30 0x7f277c0693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279815cf50 0x7f279815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27980cceb0 0x7f279815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 --2- 172.21.15.98:0/1291150048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279815cb80 0x7f27980c74f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 >> 172.21.15.98:0/1291150048 conn(0x7f2798088000 msgr2=0x7f27980c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.306+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 shutdown_connections 2023-10-06T01:39:43.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.307+0000 7f279fa82700 1 -- 172.21.15.98:0/1291150048 wait complete. 2023-10-06T01:39:43.312 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2023-10-06T01:39:43.323 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-06T01:39:43.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.445+0000 7fc420a9f700 1 Processor -- start 2023-10-06T01:39:43.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.445+0000 7fc420a9f700 1 -- start start 2023-10-06T01:39:43.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc420a9f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc41c0c4490 0x7fc41c068700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:43.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc420a9f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c0c96b0 con 0x7fc41c0b0860 2023-10-06T01:39:43.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc420a9f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c159c40 con 0x7fc41c0cc780 2023-10-06T01:39:43.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc420a9f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c0c8aa0 con 0x7fc41c0c4490 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc411d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc41c0c4490 0x7fc41c068700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc411d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc41c0c4490 0x7fc41c068700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48974/0 (socket says 172.21.15.98:48974) 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc41a59c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc41c0b0860 0x7fc41c0cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37444/0 (socket says 172.21.15.98:37444) 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.446+0000 7fc411d9b700 1 -- 172.21.15.98:0/2762746089 learned_addr learned my addr 172.21.15.98:0/2762746089 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc411d9b700 1 -- 172.21.15.98:0/2762746089 >> v1:172.21.15.184:6789/0 conn(0x7fc41c0cc780 legacy=0x7fc41c0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc411d9b700 1 -- 172.21.15.98:0/2762746089 >> v1:172.21.15.98:6789/0 conn(0x7fc41c0b0860 legacy=0x7fc41c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc411d9b700 1 -- 172.21.15.98:0/2762746089 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c0c8d80 con 0x7fc41c0c4490 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc419d9b700 1 -- 172.21.15.98:0/2762746089 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256285667 0 0) 0x7fc41c0c96b0 con 0x7fc41c0b0860 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc411d9b700 1 --2- 172.21.15.98:0/2762746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0c4490 0x7fc41c068700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4040099b0 tx=0x7fc404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419c4b7ff1d15b40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc419d9b700 1 -- 172.21.15.98:0/2762746089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400b820 con 0x7fc41c0c4490 2023-10-06T01:39:43.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc419d9b700 1 -- 172.21.15.98:0/2762746089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400b980 con 0x7fc41c0c4490 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc419d9b700 1 -- 172.21.15.98:0/2762746089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400bd20 con 0x7fc41c0c4490 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 -- 172.21.15.98:0/2762746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0c4490 msgr2=0x7fc41c068700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 --2- 172.21.15.98:0/2762746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0c4490 0x7fc41c068700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4040099b0 tx=0x7fc404009250 comp rx=0 tx=0).stop 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 -- 172.21.15.98:0/2762746089 shutdown_connections 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 --2- 172.21.15.98:0/2762746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0c4490 0x7fc41c068700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 -- 172.21.15.98:0/2762746089 >> 172.21.15.98:0/2762746089 conn(0x7fc41c088000 msgr2=0x7fc41c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:43.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 -- 172.21.15.98:0/2762746089 shutdown_connections 2023-10-06T01:39:43.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.447+0000 7fc420a9f700 1 -- 172.21.15.98:0/2762746089 wait complete. 2023-10-06T01:39:43.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 Processor -- start 2023-10-06T01:39:43.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 -- start start 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41c0b0860 0x7fc41c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc41c0c4490 0x7fc41c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c75c0 con 0x7fc41c0cc780 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c63f0 con 0x7fc41c0c4490 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc420a9f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c57e0 con 0x7fc41c0b0860 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc41ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc41ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48986/0 (socket says 172.21.15.98:48986) 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.449+0000 7fc41ad9d700 1 -- 172.21.15.98:0/3076647619 learned_addr learned my addr 172.21.15.98:0/3076647619 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41a59c700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41c0b0860 0x7fc41c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc411d9b700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc41c0c4490 0x7fc41c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:43.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 -- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41c0b0860 msgr2=0x7fc41c0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41c0b0860 0x7fc41c0c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 -- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc41c0c4490 msgr2=0x7fc41c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:43.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc41c0c4490 0x7fc41c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:43.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 -- 172.21.15.98:0/3076647619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040094f0 con 0x7fc41c0cc780 2023-10-06T01:39:43.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc41ad9d700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41400bbb0 tx=0x7fc414009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140092e0 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41400c7a0 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c0c5ac0 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400cb40 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.450+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c0c6080 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.451+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c064ad0 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.451+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc414012070 con 0x7fc41c0cc780 2023-10-06T01:39:43.457 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.452+0000 7fc412ffd700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 0x7fc3fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:43.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.452+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fc414084140 con 0x7fc41c0cc780 2023-10-06T01:39:43.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.452+0000 7fc41a59c700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 0x7fc3fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:43.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.452+0000 7fc41a59c700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 0x7fc3fc0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ca0f0 tx=0x7fc40c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:43.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.456+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc414051430 con 0x7fc41c0cc780 2023-10-06T01:39:43.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:43.638+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc41c04bb80 con 0x7fc41c0cc780 2023-10-06T01:39:44.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.309+0000 7fc412ffd700 1 -- 172.21.15.98:0/3076647619 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc41401a0d0 con 0x7fc41c0cc780 2023-10-06T01:39:44.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.314+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 msgr2=0x7fc3fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.314+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 0x7fc3fc0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ca0f0 tx=0x7fc40c009040 comp rx=0 tx=0).stop 2023-10-06T01:39:44.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.314+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 msgr2=0x7fc41c068d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.314+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41400bbb0 tx=0x7fc414009840 comp rx=0 tx=0).stop 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 shutdown_connections 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc3fc066f30 0x7fc3fc0693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc41c0cc780 0x7fc41c068d60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41c0b0860 0x7fc41c0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 --2- 172.21.15.98:0/3076647619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc41c0c4490 0x7fc41c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.317+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 >> 172.21.15.98:0/3076647619 conn(0x7fc41c088000 msgr2=0x7fc41c158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:44.321 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.318+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 shutdown_connections 2023-10-06T01:39:44.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.320+0000 7fc420a9f700 1 -- 172.21.15.98:0/3076647619 wait complete. 2023-10-06T01:39:44.328 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2023-10-06T01:39:44.342 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T01:39:44.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 Processor -- start 2023-10-06T01:39:44.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 -- start start 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc834159710 0x7fc8340cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340afbe0 con 0x7fc834159340 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340cbed0 con 0x7fc8340cba70 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.459+0000 7fc83ac97700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8340cf820 con 0x7fc834159710 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc834159710 0x7fc8340cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc834159710 0x7fc8340cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48994/0 (socket says 172.21.15.98:48994) 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 -- 172.21.15.98:0/2637881801 learned_addr learned my addr 172.21.15.98:0/2637881801 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc838a33700 1 --1- 172.21.15.98:0/2637881801 >> v1:172.21.15.98:6789/0 conn(0x7fc834159340 0x7fc8340af4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37452/0 (socket says 172.21.15.98:37452) 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 -- 172.21.15.98:0/2637881801 >> v1:172.21.15.184:6789/0 conn(0x7fc8340cba70 legacy=0x7fc8340c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 -- 172.21.15.98:0/2637881801 >> v1:172.21.15.98:6789/0 conn(0x7fc834159340 legacy=0x7fc8340af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 -- 172.21.15.98:0/2637881801 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340cfb00 con 0x7fc834159710 2023-10-06T01:39:44.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc8337fe700 1 -- 172.21.15.98:0/2637881801 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992865553 0 0) 0x7fc8240011d0 con 0x7fc834159340 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc833fff700 1 --2- 172.21.15.98:0/2637881801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159710 0x7fc8340cb530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f20 tx=0x7fc828009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df67aa5920268005 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc8337fe700 1 -- 172.21.15.98:0/2637881801 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800e070 con 0x7fc834159710 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc8337fe700 1 -- 172.21.15.98:0/2637881801 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800ba40 con 0x7fc834159710 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.460+0000 7fc8337fe700 1 -- 172.21.15.98:0/2637881801 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280092e0 con 0x7fc834159710 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 -- 172.21.15.98:0/2637881801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159710 msgr2=0x7fc8340cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 --2- 172.21.15.98:0/2637881801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159710 0x7fc8340cb530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f20 tx=0x7fc828009840 comp rx=0 tx=0).stop 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 -- 172.21.15.98:0/2637881801 shutdown_connections 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 --2- 172.21.15.98:0/2637881801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159710 0x7fc8340cb530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 -- 172.21.15.98:0/2637881801 >> 172.21.15.98:0/2637881801 conn(0x7fc834099720 msgr2=0x7fc83406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 -- 172.21.15.98:0/2637881801 shutdown_connections 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.461+0000 7fc83ac97700 1 -- 172.21.15.98:0/2637881801 wait complete. 2023-10-06T01:39:44.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.462+0000 7fc83ac97700 1 Processor -- start 2023-10-06T01:39:44.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.462+0000 7fc83ac97700 1 -- start start 2023-10-06T01:39:44.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8340cba70 0x7fc8340c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:44.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc834159710 0x7fc83416d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8341575c0 con 0x7fc834159340 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834156c90 con 0x7fc8340cba70 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc834156f70 con 0x7fc834159710 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49010/0 (socket says 172.21.15.98:49010) 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 -- 172.21.15.98:0/1863548243 learned_addr learned my addr 172.21.15.98:0/1863548243 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc839234700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc834159710 0x7fc83416d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc838a33700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8340cba70 0x7fc8340c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 -- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc834159710 msgr2=0x7fc83416d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc834159710 0x7fc83416d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 -- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8340cba70 msgr2=0x7fc8340c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8340cba70 0x7fc8340c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 -- 172.21.15.98:0/1863548243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009ae0 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc833fff700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b6c0 tx=0x7fc828017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828007000 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8280129a0 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828012d60 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc834157250 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.463+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8340b4df0 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.464+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc834157730 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.465+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc82801e640 con 0x7fc834159340 2023-10-06T01:39:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.465+0000 7fc830ff9700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 0x7fc8200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.465+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fc828085e80 con 0x7fc834159340 2023-10-06T01:39:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.465+0000 7fc838a33700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 0x7fc8200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.466+0000 7fc838a33700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 0x7fc8200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc824002890 tx=0x7fc824009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.470+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc828052ff0 con 0x7fc834159340 2023-10-06T01:39:44.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:44.694+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc8340cbed0 con 0x7fc834159340 2023-10-06T01:39:45.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.313+0000 7fc830ff9700 1 -- 172.21.15.98:0/1863548243 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc8340cbed0 con 0x7fc834159340 2023-10-06T01:39:45.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.331+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 msgr2=0x7fc8200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.331+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 0x7fc8200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc824002890 tx=0x7fc824009040 comp rx=0 tx=0).stop 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.331+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 msgr2=0x7fc834159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.331+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b6c0 tx=0x7fc828017000 comp rx=0 tx=0).stop 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.331+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 shutdown_connections 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc820066f30 0x7fc8200693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc834159710 0x7fc83416d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8340cba70 0x7fc8340c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 --2- 172.21.15.98:0/1863548243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834159340 0x7fc834159bb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 >> 172.21.15.98:0/1863548243 conn(0x7fc834099720 msgr2=0x7fc8340688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:45.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 shutdown_connections 2023-10-06T01:39:45.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.332+0000 7fc83ac97700 1 -- 172.21.15.98:0/1863548243 wait complete. 2023-10-06T01:39:45.341 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T01:39:45.355 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-06T01:39:45.355 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-06T01:39:45.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.476+0000 7f093f531700 1 Processor -- start 2023-10-06T01:39:45.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.476+0000 7f093f531700 1 -- start start 2023-10-06T01:39:45.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093f531700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093f531700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f093806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093f531700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093806b330 con 0x7f0938155f60 2023-10-06T01:39:45.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093f531700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09380afb10 con 0x7f09380af670 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093f531700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0938068f80 con 0x7f0938068b40 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093dd2e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093ed30700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f093806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093e52f700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f0938155f60 0x7f09380c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:55430/0 (socket says 172.21.15.98:55430) 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093dd2e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60084/0 (socket says 172.21.15.98:60084) 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.477+0000 7f093e52f700 1 -- 172.21.15.98:0/3190963027 learned_addr learned my addr 172.21.15.98:0/3190963027 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093ed30700 1 -- 172.21.15.98:0/3190963027 >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 msgr2=0x7f0938068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093ed30700 1 --2- 172.21.15.98:0/3190963027 >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093ed30700 1 -- 172.21.15.98:0/3190963027 >> v1:172.21.15.184:6789/0 conn(0x7f0938155f60 legacy=0x7f09380c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093ed30700 1 -- 172.21.15.98:0/3190963027 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380c77c0 con 0x7f0938068b40 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093dd2e700 1 --2- 172.21.15.98:0/3190963027 >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093ed30700 1 --2- 172.21.15.98:0/3190963027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f093806b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09300098e0 tx=0x7f09300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11af00bdaa1f844 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093d52d700 1 -- 172.21.15.98:0/3190963027 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000b640 con 0x7f0938068b40 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093d52d700 1 -- 172.21.15.98:0/3190963027 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000b7a0 con 0x7f0938068b40 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093d52d700 1 -- 172.21.15.98:0/3190963027 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000bb40 con 0x7f0938068b40 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.478+0000 7f093f531700 1 -- 172.21.15.98:0/3190963027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 msgr2=0x7f093806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 --2- 172.21.15.98:0/3190963027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f093806b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09300098e0 tx=0x7f09300091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 -- 172.21.15.98:0/3190963027 shutdown_connections 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 --2- 172.21.15.98:0/3190963027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f093806b7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 --2- 172.21.15.98:0/3190963027 >> v2:172.21.15.184:3300/0 conn(0x7f09380af670 0x7f0938068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 -- 172.21.15.98:0/3190963027 >> 172.21.15.98:0/3190963027 conn(0x7f09380c9ca0 msgr2=0x7f093806da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 -- 172.21.15.98:0/3190963027 shutdown_connections 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.479+0000 7f093f531700 1 -- 172.21.15.98:0/3190963027 wait complete. 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 Processor -- start 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 -- start start 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f09381550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0938155f60 0x7f09380b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938155630 con 0x7f09380af670 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938156700 con 0x7f0938155f60 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.480+0000 7f093f531700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380ae6b0 con 0x7f0938068b40 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49014/0 (socket says 172.21.15.98:49014) 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 -- 172.21.15.98:0/1754741043 learned_addr learned my addr 172.21.15.98:0/1754741043 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093ed30700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0938155f60 0x7f09380b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093e52f700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f09381550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 -- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 msgr2=0x7f09381550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f09381550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 -- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0938155f60 msgr2=0x7f09380b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0938155f60 0x7f09380b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 -- 172.21.15.98:0/1754741043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928009420 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093dd2e700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f092800ecb0 tx=0x7f09280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800ee60 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092800cbe0 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928013470 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0930009420 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.481+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0938159190 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.482+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0938156960 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.483+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09280137e0 con 0x7f09380af670 2023-10-06T01:39:45.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.483+0000 7f092e7fc700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 0x7f091c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:45.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.483+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0928084a90 con 0x7f09380af670 2023-10-06T01:39:45.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.483+0000 7f093e52f700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 0x7f091c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:45.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.483+0000 7f093e52f700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 0x7f091c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f0934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:45.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.487+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0938156960 con 0x7f09380af670 2023-10-06T01:39:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:45.681+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0938156960 con 0x7f09380af670 2023-10-06T01:39:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.331+0000 7f092e7fc700 1 -- 172.21.15.98:0/1754741043 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0938156960 con 0x7f09380af670 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 msgr2=0x7f091c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 0x7f091c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f0934009040 comp rx=0 tx=0).stop 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 msgr2=0x7f09381589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f092800ecb0 tx=0x7f09280091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 shutdown_connections 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f091c066f30 0x7f091c0693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0938155f60 0x7f09380b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938068b40 0x7f09381550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 --2- 172.21.15.98:0/1754741043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09380af670 0x7f09381589a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 >> 172.21.15.98:0/1754741043 conn(0x7f09380c9ca0 msgr2=0x7f09380cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:46.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 shutdown_connections 2023-10-06T01:39:46.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.334+0000 7f093f531700 1 -- 172.21.15.98:0/1754741043 wait complete. 2023-10-06T01:39:46.350 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T01:39:46.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.475+0000 7f1aeba9a700 1 Processor -- start 2023-10-06T01:39:46.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.475+0000 7f1aeba9a700 1 -- start start 2023-10-06T01:39:46.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ae4159340 0x7f1ae40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1ae4159710 0x7f1ae40cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae40cbed0 con 0x7f1ae4159710 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae40cf820 con 0x7f1ae4159340 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aeba9a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40cfb00 con 0x7f1ae40cba70 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1ae4159710 0x7f1ae40cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1ae4159710 0x7f1ae40cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49030/0 (socket says 172.21.15.98:49030) 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 -- 172.21.15.98:0/3855675060 learned_addr learned my addr 172.21.15.98:0/3855675060 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1aea037700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9836700 1 --2- 172.21.15.98:0/3855675060 >> v2:172.21.15.184:3300/0 conn(0x7f1ae4159340 0x7f1ae40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 -- 172.21.15.98:0/3855675060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 msgr2=0x7f1ae40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:46.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 -- 172.21.15.98:0/3855675060 >> v2:172.21.15.184:3300/0 conn(0x7f1ae4159340 msgr2=0x7f1ae40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 --2- 172.21.15.98:0/3855675060 >> v2:172.21.15.184:3300/0 conn(0x7f1ae4159340 0x7f1ae40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.476+0000 7f1ae9035700 1 -- 172.21.15.98:0/3855675060 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40cfde0 con 0x7f1ae4159710 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1ae9035700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae40cb530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4071290 tx=0x7f1ad8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0d219988000298 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1ae8834700 1 -- 172.21.15.98:0/3855675060 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800b640 con 0x7f1ae4159710 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1ae8834700 1 -- 172.21.15.98:0/3855675060 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad800b7a0 con 0x7f1ae4159710 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1ae8834700 1 -- 172.21.15.98:0/3855675060 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800bb60 con 0x7f1ae4159710 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 -- 172.21.15.98:0/3855675060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 msgr2=0x7f1ae40cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae40cb530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4071290 tx=0x7f1ad8009250 comp rx=0 tx=0).stop 2023-10-06T01:39:46.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 -- 172.21.15.98:0/3855675060 shutdown_connections 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/3855675060 >> v2:172.21.15.184:3300/0 conn(0x7f1ae4159340 0x7f1ae40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/3855675060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae40cb530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 -- 172.21.15.98:0/3855675060 >> 172.21.15.98:0/3855675060 conn(0x7f1ae4099720 msgr2=0x7f1ae406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 -- 172.21.15.98:0/3855675060 shutdown_connections 2023-10-06T01:39:46.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.477+0000 7f1aeba9a700 1 -- 172.21.15.98:0/3855675060 wait complete. 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.478+0000 7f1aeba9a700 1 Processor -- start 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 -- start start 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae4159340 0x7f1ae40c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416a5f0 con 0x7f1ae4159710 2023-10-06T01:39:46.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416a160 con 0x7f1ae4159340 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aeba9a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40c9d60 con 0x7f1ae40cba70 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aea037700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aea037700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49042/0 (socket says 172.21.15.98:49042) 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1aea037700 1 -- 172.21.15.98:0/1934816025 learned_addr learned my addr 172.21.15.98:0/1934816025 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1ae9836700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.479+0000 7f1ae9035700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae4159340 0x7f1ae40c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 -- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 msgr2=0x7f1ae41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae41540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 -- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae4159340 msgr2=0x7f1ae40c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae4159340 0x7f1ae40c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 -- 172.21.15.98:0/1934816025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80094f0 con 0x7f1ae4159710 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1ae9836700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aea037700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009ef0 tx=0x7f1adc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00cb70 con 0x7f1ae4159710 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc00f040 con 0x7f1ae4159710 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae4169830 con 0x7f1ae4159710 2023-10-06T01:39:46.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc014480 con 0x7f1ae4159710 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.480+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae4169df0 con 0x7f1ae4159710 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.481+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae40696e0 con 0x7f1ae4159710 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.481+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1adc014830 con 0x7f1ae4159710 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.482+0000 7f1ae1ffb700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 0x7f1ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.482+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f1adc085070 con 0x7f1ae4159710 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.482+0000 7f1ae9836700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 0x7f1ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:46.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.482+0000 7f1ae9836700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 0x7f1ad00693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009fd0 tx=0x7f1ad4009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:46.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.486+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1adc0520a0 con 0x7f1ae4159710 2023-10-06T01:39:46.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:46.670+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1ae4064ad0 con 0x7f1ae4159710 2023-10-06T01:39:48.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.335+0000 7f1ae1ffb700 1 -- 172.21.15.98:0/1934816025 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1adc0556c0 con 0x7f1ae4159710 2023-10-06T01:39:48.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.339+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 msgr2=0x7f1ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.339+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 0x7f1ad00693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009fd0 tx=0x7f1ad4009120 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.339+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 msgr2=0x7f1ae4157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.339+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009ef0 tx=0x7f1adc00b040 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 shutdown_connections 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1ad0066f30 0x7f1ad00693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4159710 0x7f1ae4157990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ae40cba70 0x7f1ae41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 --2- 172.21.15.98:0/1934816025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ae4159340 0x7f1ae40c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 >> 172.21.15.98:0/1934816025 conn(0x7f1ae4099720 msgr2=0x7f1ae4068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 shutdown_connections 2023-10-06T01:39:48.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.340+0000 7f1aeba9a700 1 -- 172.21.15.98:0/1934816025 wait complete. 2023-10-06T01:39:48.350 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data_ec' created 2023-10-06T01:39:48.363 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-06T01:39:48.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 Processor -- start 2023-10-06T01:39:48.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 -- start start 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95c006ed20 0x7f95c00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 0x7f95c00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00b2d20 con 0x7f95c00aee20 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c006fb80 con 0x7f95c006ed20 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.485+0000 7f95c5797700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00b0450 con 0x7f95c006f0f0 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95c006ed20 0x7f95c00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bf7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f95c00aee20 0x7f95c00af280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37464/0 (socket says 172.21.15.98:37464) 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bf7fe700 1 -- 172.21.15.98:0/3217024489 learned_addr learned my addr 172.21.15.98:0/3217024489 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:48.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95be7fc700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 0x7f95c00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 -- 172.21.15.98:0/3217024489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 msgr2=0x7f95c00b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 0x7f95c00b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 -- 172.21.15.98:0/3217024489 >> v1:172.21.15.98:6789/0 conn(0x7f95c00aee20 legacy=0x7f95c00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 -- 172.21.15.98:0/3217024489 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00b2a40 con 0x7f95c006ed20 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95be7fc700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 0x7f95c00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bdffb700 1 -- 172.21.15.98:0/3217024489 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736399374 0 0) 0x7f95c00b2d20 con 0x7f95c00aee20 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95beffd700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006ed20 0x7f95c00b23a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009910 tx=0x7f95b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fa82b0c34b62cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bdffb700 1 -- 172.21.15.98:0/3217024489 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000b820 con 0x7f95c006ed20 2023-10-06T01:39:48.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bdffb700 1 -- 172.21.15.98:0/3217024489 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b000b980 con 0x7f95c006ed20 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.486+0000 7f95bdffb700 1 -- 172.21.15.98:0/3217024489 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000bd20 con 0x7f95c006ed20 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 -- 172.21.15.98:0/3217024489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006ed20 msgr2=0x7f95c00b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006ed20 0x7f95c00b23a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009910 tx=0x7f95b00091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 -- 172.21.15.98:0/3217024489 shutdown_connections 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006ed20 0x7f95c00b23a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 --2- 172.21.15.98:0/3217024489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c006f0f0 0x7f95c00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 -- 172.21.15.98:0/3217024489 >> 172.21.15.98:0/3217024489 conn(0x7f95c0068330 msgr2=0x7f95c0068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 -- 172.21.15.98:0/3217024489 shutdown_connections 2023-10-06T01:39:48.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.487+0000 7f95c5797700 1 -- 172.21.15.98:0/3217024489 wait complete. 2023-10-06T01:39:48.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.488+0000 7f95c5797700 1 Processor -- start 2023-10-06T01:39:48.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.488+0000 7f95c5797700 1 -- start start 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c006ed20 0x7f95c0169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c00aee20 0x7f95c0155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00c9c70 con 0x7f95c006f0f0 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00c9250 con 0x7f95c006ed20 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0154270 con 0x7f95c00aee20 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49060/0 (socket says 172.21.15.98:49060) 2023-10-06T01:39:48.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 -- 172.21.15.98:0/3120064519 learned_addr learned my addr 172.21.15.98:0/3120064519 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95beffd700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c006ed20 0x7f95c0169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95bf7fe700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c00aee20 0x7f95c0155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 -- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c00aee20 msgr2=0x7f95c0155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c00aee20 0x7f95c0155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 -- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c006ed20 msgr2=0x7f95c0169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c006ed20 0x7f95c0169a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 -- 172.21.15.98:0/3120064519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0009420 con 0x7f95c006f0f0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95be7fc700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95b400e8d0 tx=0x7f95b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b400ee70 con 0x7f95c006f0f0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b400c5e0 con 0x7f95c006f0f0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c016a110 con 0x7f95c006f0f0 2023-10-06T01:39:48.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b400c980 con 0x7f95c006f0f0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.489+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c016a750 con 0x7f95c006f0f0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.490+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c016aa70 con 0x7f95c006f0f0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.491+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95b4013410 con 0x7f95c006f0f0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.491+0000 7f95a6ffd700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 0x7f95ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.491+0000 7f95beffd700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 0x7f95ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.491+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f95b4084770 con 0x7f95c006f0f0 2023-10-06T01:39:48.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.492+0000 7f95beffd700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 0x7f95ac0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95b0003080 tx=0x7f95b0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:48.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.496+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95b40515b0 con 0x7f95c006f0f0 2023-10-06T01:39:48.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:48.677+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f95c0064ad0 con 0x7f95c006f0f0 2023-10-06T01:39:49.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.337+0000 7f95a6ffd700 1 -- 172.21.15.98:0/3120064519 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f95b4054bd0 con 0x7f95c006f0f0 2023-10-06T01:39:49.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.357+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 msgr2=0x7f95ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.357+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 0x7f95ac0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95b0003080 tx=0x7f95b0017000 comp rx=0 tx=0).stop 2023-10-06T01:39:49.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.357+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 msgr2=0x7f95c00c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.357+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95b400e8d0 tx=0x7f95b4008040 comp rx=0 tx=0).stop 2023-10-06T01:39:49.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 shutdown_connections 2023-10-06T01:39:49.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f95ac066f30 0x7f95ac0693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95c00aee20 0x7f95c0155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95c006ed20 0x7f95c0169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 --2- 172.21.15.98:0/3120064519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95c006f0f0 0x7f95c00c9730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.358+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 >> 172.21.15.98:0/3120064519 conn(0x7f95c0068330 msgr2=0x7f95c01568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:49.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.360+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 shutdown_connections 2023-10-06T01:39:49.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.360+0000 7f95c5797700 1 -- 172.21.15.98:0/3120064519 wait complete. 2023-10-06T01:39:49.363 INFO:teuthology.orchestra.run.smithi098.stderr:set pool 5 allow_ec_overwrites to true 2023-10-06T01:39:49.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-06T01:39:49.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.492+0000 7fcbeaa65700 1 Processor -- start 2023-10-06T01:39:49.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.492+0000 7fcbeaa65700 1 -- start start 2023-10-06T01:39:49.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbeaa65700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcbe406a090 0x7fcbe406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbeaa65700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbe415a5f0 0x7fcbe406be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbeaa65700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe415d170 con 0x7fcbe406a460 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbeaa65700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe415a9f0 con 0x7fcbe406a090 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbeaa65700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe415d3c0 con 0x7fcbe415a5f0 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbe3fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcbe406a460 0x7fcbe4159ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37466/0 (socket says 172.21.15.98:37466) 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbe3fff700 1 -- 172.21.15.98:0/1482941457 learned_addr learned my addr 172.21.15.98:0/1482941457 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbe8801700 1 --2- 172.21.15.98:0/1482941457 >> v2:172.21.15.98:3300/0 conn(0x7fcbe406a090 0x7fcbe406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.493+0000 7fcbe9002700 1 --2- 172.21.15.98:0/1482941457 >> v2:172.21.15.184:3300/0 conn(0x7fcbe415a5f0 0x7fcbe406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe37fe700 1 -- 172.21.15.98:0/1482941457 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402341018 0 0) 0x7fcbe415d170 con 0x7fcbe406a460 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe37fe700 1 -- 172.21.15.98:0/1482941457 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd0003570 con 0x7fcbe406a460 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe8801700 1 -- 172.21.15.98:0/1482941457 >> v2:172.21.15.184:3300/0 conn(0x7fcbe415a5f0 msgr2=0x7fcbe406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:49.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe8801700 1 --2- 172.21.15.98:0/1482941457 >> v2:172.21.15.184:3300/0 conn(0x7fcbe415a5f0 0x7fcbe406be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe8801700 1 -- 172.21.15.98:0/1482941457 >> v1:172.21.15.98:6789/0 conn(0x7fcbe406a460 legacy=0x7fcbe4159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe8801700 1 -- 172.21.15.98:0/1482941457 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe415d6a0 con 0x7fcbe406a090 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe8801700 1 --2- 172.21.15.98:0/1482941457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a090 0x7fcbe406b8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40c52d0 tx=0x7fcbd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e10977e8e62c30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe37fe700 1 -- 172.21.15.98:0/1482941457 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400e070 con 0x7fcbe406a090 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe37fe700 1 -- 172.21.15.98:0/1482941457 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400bab0 con 0x7fcbe406a090 2023-10-06T01:39:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.494+0000 7fcbe37fe700 1 -- 172.21.15.98:0/1482941457 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400be50 con 0x7fcbe406a090 2023-10-06T01:39:49.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 -- 172.21.15.98:0/1482941457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a090 msgr2=0x7fcbe406b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/1482941457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a090 0x7fcbe406b8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40c52d0 tx=0x7fcbd4009780 comp rx=0 tx=0).stop 2023-10-06T01:39:49.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 -- 172.21.15.98:0/1482941457 shutdown_connections 2023-10-06T01:39:49.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/1482941457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a090 0x7fcbe406b8e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/1482941457 >> v2:172.21.15.184:3300/0 conn(0x7fcbe415a5f0 0x7fcbe406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 -- 172.21.15.98:0/1482941457 >> 172.21.15.98:0/1482941457 conn(0x7fcbe40ce330 msgr2=0x7fcbe40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:49.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 -- 172.21.15.98:0/1482941457 shutdown_connections 2023-10-06T01:39:49.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.495+0000 7fcbeaa65700 1 -- 172.21.15.98:0/1482941457 wait complete. 2023-10-06T01:39:49.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.496+0000 7fcbeaa65700 1 Processor -- start 2023-10-06T01:39:49.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.496+0000 7fcbeaa65700 1 -- start start 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbe406a090 0x7fcbe41692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbe415a5f0 0x7fcbe40c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40b1cb0 con 0x7fcbe406a460 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40b0960 con 0x7fcbe406a090 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe41698f0 con 0x7fcbe415a5f0 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49078/0 (socket says 172.21.15.98:49078) 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 -- 172.21.15.98:0/2629234365 learned_addr learned my addr 172.21.15.98:0/2629234365 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe8801700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbe406a090 0x7fcbe41692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe9002700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbe415a5f0 0x7fcbe40c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 -- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbe415a5f0 msgr2=0x7fcbe40c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbe415a5f0 0x7fcbe40c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 -- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbe406a090 msgr2=0x7fcbe41692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbe406a090 0x7fcbe41692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 -- 172.21.15.98:0/2629234365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009a20 con 0x7fcbe406a460 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe3fff700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c7c0 tx=0x7fcbcc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc005c00 con 0x7fcbe406a460 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc005d60 con 0x7fcbe406a460 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe4169bd0 con 0x7fcbe406a460 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0072e0 con 0x7fcbe406a460 2023-10-06T01:39:49.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.497+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe4169f60 con 0x7fcbe406a460 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.498+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe40b1df0 con 0x7fcbe406a460 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.499+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbcc007650 con 0x7fcbe406a460 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.499+0000 7fcbe0ff9700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 0x7fcbc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.499+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fcbcc083ce0 con 0x7fcbe406a460 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.499+0000 7fcbe8801700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 0x7fcbc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.499+0000 7fcbe8801700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 0x7fcbc40693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40099f0 tx=0x7fcbd4019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:49.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.503+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbcc04b5a0 con 0x7fcbe406a460 2023-10-06T01:39:49.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:49.730+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fcbe4064ad0 con 0x7fcbe406a460 2023-10-06T01:39:50.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:39:50.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:39:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.371+0000 7fcbe0ff9700 1 -- 172.21.15.98:0/2629234365 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcbcc017020 con 0x7fcbe406a460 2023-10-06T01:39:50.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 msgr2=0x7fcbc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 0x7fcbc40693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40099f0 tx=0x7fcbd4019000 comp rx=0 tx=0).stop 2023-10-06T01:39:50.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 msgr2=0x7fcbe40b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c7c0 tx=0x7fcbcc007d90 comp rx=0 tx=0).stop 2023-10-06T01:39:50.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 shutdown_connections 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcbc4066f30 0x7fcbc40693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbe415a5f0 0x7fcbe40c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbe406a090 0x7fcbe41692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 --2- 172.21.15.98:0/2629234365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe406a460 0x7fcbe40b0420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 >> 172.21.15.98:0/2629234365 conn(0x7fcbe40ce330 msgr2=0x7fcbe40cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 shutdown_connections 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.373+0000 7fcbeaa65700 1 -- 172.21.15.98:0/2629234365 wait complete. 2023-10-06T01:39:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:added data pool 5 to fsmap 2023-10-06T01:39:50.387 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:39:50.413 INFO:tasks.ceph.mds.c.smithi098.stderr:2023-10-06T01:39:50.412+0000 7f03b850f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:39:50.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.499+0000 7f03df5fe700 1 Processor -- start 2023-10-06T01:39:50.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.499+0000 7f03df5fe700 1 -- start start 2023-10-06T01:39:50.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03df5fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f03d80b29f0 0x7f03d8159160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03df5fe700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80b0de0 con 0x7f03d80b2620 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03df5fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80cbfe0 con 0x7f03d81596a0 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03df5fe700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d80cb3d0 con 0x7f03d80b29f0 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03dcb99700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f03d80b29f0 0x7f03d8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03dd39a700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f03d80b2620 0x7f03d80b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37482/0 (socket says 172.21.15.98:37482) 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.500+0000 7f03dd39a700 1 -- 172.21.15.98:0/4090600029 learned_addr learned my addr 172.21.15.98:0/4090600029 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03dcb99700 1 -- 172.21.15.98:0/4090600029 >> v1:172.21.15.184:6789/0 conn(0x7f03d81596a0 legacy=0x7f03d80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03dcb99700 1 -- 172.21.15.98:0/4090600029 >> v1:172.21.15.98:6789/0 conn(0x7f03d80b2620 legacy=0x7f03d80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03d7fff700 1 -- 172.21.15.98:0/4090600029 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815370124 0 0) 0x7f03c8001610 con 0x7f03d80b2620 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03dcb99700 1 -- 172.21.15.98:0/4090600029 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80cb6b0 con 0x7f03d80b29f0 2023-10-06T01:39:50.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03dcb99700 1 --2- 172.21.15.98:0/4090600029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b29f0 0x7f03d8159160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098e0 tx=0x7f03cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b1aefe5be9de88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03d7fff700 1 -- 172.21.15.98:0/4090600029 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc009e80 con 0x7f03d80b29f0 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03d7fff700 1 -- 172.21.15.98:0/4090600029 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00b860 con 0x7f03d80b29f0 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03d7fff700 1 -- 172.21.15.98:0/4090600029 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00bbe0 con 0x7f03d80b29f0 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03df5fe700 1 -- 172.21.15.98:0/4090600029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b29f0 msgr2=0x7f03d8159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03df5fe700 1 --2- 172.21.15.98:0/4090600029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b29f0 0x7f03d8159160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0098e0 tx=0x7f03cc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03df5fe700 1 -- 172.21.15.98:0/4090600029 shutdown_connections 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03df5fe700 1 --2- 172.21.15.98:0/4090600029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b29f0 0x7f03d8159160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.501+0000 7f03df5fe700 1 -- 172.21.15.98:0/4090600029 >> 172.21.15.98:0/4090600029 conn(0x7f03d806ed20 msgr2=0x7f03d806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.502+0000 7f03df5fe700 1 -- 172.21.15.98:0/4090600029 shutdown_connections 2023-10-06T01:39:50.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.502+0000 7f03df5fe700 1 -- 172.21.15.98:0/4090600029 wait complete. 2023-10-06T01:39:50.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 Processor -- start 2023-10-06T01:39:50.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 -- start start 2023-10-06T01:39:50.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03d80b29f0 0x7f03d80c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 0x7f03d8150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815e760 con 0x7f03d80b2620 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d816e5a0 con 0x7f03d81596a0 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.503+0000 7f03df5fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d80c8080 con 0x7f03d80b29f0 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49086/0 (socket says 172.21.15.98:49086) 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 -- 172.21.15.98:0/1754409826 learned_addr learned my addr 172.21.15.98:0/1754409826 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03ddb9b700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 0x7f03d8150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dcb99700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03d80b29f0 0x7f03d80c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 -- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03d80b29f0 msgr2=0x7f03d80c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03d80b29f0 0x7f03d80c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 -- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 msgr2=0x7f03d8150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 0x7f03d8150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 -- 172.21.15.98:0/1754409826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc009420 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03ddb9b700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 0x7f03d8150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03dd39a700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c980 tx=0x7f03c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8005d80 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c800a570 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d815e480 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800a930 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.504+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d80b36c0 con 0x7f03d80b2620 2023-10-06T01:39:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.505+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d815e8a0 con 0x7f03d80b2620 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.505+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03c800ad20 con 0x7f03d80b2620 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.506+0000 7f03d57fa700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 0x7f03c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.506+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f03c80845b0 con 0x7f03d80b2620 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.506+0000 7f03dcb99700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 0x7f03c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.506+0000 7f03dcb99700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 0x7f03c40693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0093f0 tx=0x7f03cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.510+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03c8051450 con 0x7f03d80b2620 2023-10-06T01:39:50.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.733+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03d80b0de0 con 0x7f03d80b2620 2023-10-06T01:39:50.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.733+0000 7f03d57fa700 1 -- 172.21.15.98:0/1754409826 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f03d80b0de0 con 0x7f03d80b2620 2023-10-06T01:39:50.735 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:50.735 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:50.365301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T01:39:50.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 msgr2=0x7f03c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 0x7f03c40693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0093f0 tx=0x7f03cc017000 comp rx=0 tx=0).stop 2023-10-06T01:39:50.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 msgr2=0x7f03d816e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c980 tx=0x7f03c8007e00 comp rx=0 tx=0).stop 2023-10-06T01:39:50.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 shutdown_connections 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f03c4066f30 0x7f03c40693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d81596a0 0x7f03d8150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d80b2620 0x7f03d816e060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 --2- 172.21.15.98:0/1754409826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03d80b29f0 0x7f03d80c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 >> 172.21.15.98:0/1754409826 conn(0x7f03d806ed20 msgr2=0x7f03d80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 shutdown_connections 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.735+0000 7f03df5fe700 1 -- 172.21.15.98:0/1754409826 wait complete. 2023-10-06T01:39:50.738 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2023-10-06T01:39:50.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:50.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.903+0000 7f1601f5f700 1 Processor -- start 2023-10-06T01:39:50.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.903+0000 7f1601f5f700 1 -- start start 2023-10-06T01:39:50.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f1601f5f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 0x7f15fc0aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f1601f5f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f15fc0b07a0 0x7f15fc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f1601f5f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc0b0ca0 con 0x7f15fc1595a0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f1601f5f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0b0f80 con 0x7f15fc0b07a0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f1601f5f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0b1260 con 0x7f15fc0b49e0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f15fb7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f15fc1595a0 0x7f15fc0aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37492/0 (socket says 172.21.15.98:37492) 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f15fb7fe700 1 -- 172.21.15.98:0/543552269 learned_addr learned my addr 172.21.15.98:0/543552269 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.904+0000 7f15fbfff700 1 --2- 172.21.15.98:0/543552269 >> v2:172.21.15.98:3300/0 conn(0x7f15fc0b07a0 0x7f15fc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fa7fc700 1 -- 172.21.15.98:0/543552269 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184887269 0 0) 0x7f15fc0b0ca0 con 0x7f15fc1595a0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15faffd700 1 --2- 172.21.15.98:0/543552269 >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 0x7f15fc0aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fa7fc700 1 -- 172.21.15.98:0/543552269 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e4003570 con 0x7f15fc1595a0 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fbfff700 1 -- 172.21.15.98:0/543552269 >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 msgr2=0x7f15fc0aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fbfff700 1 --2- 172.21.15.98:0/543552269 >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 0x7f15fc0aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fbfff700 1 -- 172.21.15.98:0/543552269 >> v1:172.21.15.98:6789/0 conn(0x7f15fc1595a0 legacy=0x7f15fc0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fbfff700 1 -- 172.21.15.98:0/543552269 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0b1540 con 0x7f15fc0b07a0 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15faffd700 1 --2- 172.21.15.98:0/543552269 >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 0x7f15fc0aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fbfff700 1 --2- 172.21.15.98:0/543552269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b07a0 0x7f15fc0b1c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009e60 tx=0x7f15f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2612dec2a99bdf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fa7fc700 1 -- 172.21.15.98:0/543552269 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400e070 con 0x7f15fc0b07a0 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fa7fc700 1 -- 172.21.15.98:0/543552269 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f400bb20 con 0x7f15fc0b07a0 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f15fa7fc700 1 -- 172.21.15.98:0/543552269 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400bea0 con 0x7f15fc0b07a0 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 -- 172.21.15.98:0/543552269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b07a0 msgr2=0x7f15fc0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 --2- 172.21.15.98:0/543552269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b07a0 0x7f15fc0b1c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009e60 tx=0x7f15f4009780 comp rx=0 tx=0).stop 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 -- 172.21.15.98:0/543552269 shutdown_connections 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 --2- 172.21.15.98:0/543552269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b07a0 0x7f15fc0b1c00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 --2- 172.21.15.98:0/543552269 >> v2:172.21.15.184:3300/0 conn(0x7f15fc0b49e0 0x7f15fc0aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 -- 172.21.15.98:0/543552269 >> 172.21.15.98:0/543552269 conn(0x7f15fc099830 msgr2=0x7f15fc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:50.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 -- 172.21.15.98:0/543552269 shutdown_connections 2023-10-06T01:39:50.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.905+0000 7f1601f5f700 1 -- 172.21.15.98:0/543552269 wait complete. 2023-10-06T01:39:50.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 Processor -- start 2023-10-06T01:39:50.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 -- start start 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15fc0b07a0 0x7f15fc159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15fc1595a0 0x7f15fc169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15aef0 con 0x7f15fc0b49e0 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15b310 con 0x7f15fc0b07a0 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f1601f5f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15a270 con 0x7f15fc1595a0 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f15faffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f15faffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49104/0 (socket says 172.21.15.98:49104) 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.907+0000 7f15faffd700 1 -- 172.21.15.98:0/1651655158 learned_addr learned my addr 172.21.15.98:0/1651655158 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15fb7fe700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15fc0b07a0 0x7f15fc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15fbfff700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15fc1595a0 0x7f15fc169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 -- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15fc1595a0 msgr2=0x7f15fc169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15fc1595a0 0x7f15fc169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 -- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15fc0b07a0 msgr2=0x7f15fc159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15fc0b07a0 0x7f15fc159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 -- 172.21.15.98:0/1651655158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009a20 con 0x7f15fc0b49e0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15faffd700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c800 tx=0x7f15f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0011600 con 0x7f15fc0b49e0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0011760 con 0x7f15fc0b49e0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc15a550 con 0x7f15fc0b49e0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0011b20 con 0x7f15fc0b49e0 2023-10-06T01:39:50.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.908+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc15ab10 con 0x7f15fc0b49e0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.909+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc064ad0 con 0x7f15fc0b49e0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.909+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15f0018360 con 0x7f15fc0b49e0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.910+0000 7f15e37fe700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 0x7f15e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.910+0000 7f15fb7fe700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 0x7f15e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.910+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f15f0084740 con 0x7f15fc0b49e0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.910+0000 7f15fb7fe700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 0x7f15e80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15ec000c00 tx=0x7f15ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:50.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:50.914+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15f0054f50 con 0x7f15fc0b49e0 2023-10-06T01:39:51.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.099+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15fc0b1820 con 0x7f15fc0b49e0 2023-10-06T01:39:51.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.100+0000 7f15e37fe700 1 -- 172.21.15.98:0/1651655158 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f15f0054ae0 con 0x7f15fc0b49e0 2023-10-06T01:39:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:51.102 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:51.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 msgr2=0x7f15e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 0x7f15e80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15ec000c00 tx=0x7f15ec009040 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 msgr2=0x7f15fc169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c800 tx=0x7f15f0008040 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 shutdown_connections 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f15e8066f30 0x7f15e80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15fc1595a0 0x7f15fc169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15fc0b07a0 0x7f15fc159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 --2- 172.21.15.98:0/1651655158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15fc0b49e0 0x7f15fc169520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 >> 172.21.15.98:0/1651655158 conn(0x7f15fc099830 msgr2=0x7f15fc068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 shutdown_connections 2023-10-06T01:39:51.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.103+0000 7f1601f5f700 1 -- 172.21.15.98:0/1651655158 wait complete. 2023-10-06T01:39:51.117 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:51.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.229+0000 7fd4f8b32700 1 Processor -- start 2023-10-06T01:39:51.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.229+0000 7fd4f8b32700 1 -- start start 2023-10-06T01:39:51.238 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f8b32700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4f40c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f8b32700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40cdb80 con 0x7fd4f40c8d90 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f8b32700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40c91d0 con 0x7fd4f415a8b0 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f8b32700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40ccf70 con 0x7fd4f415ac80 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f2d9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd4f40c8d90 0x7fd4f40ab400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37508/0 (socket says 172.21.15.98:37508) 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/3234244217 learned_addr learned my addr 172.21.15.98:0/3234244217 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f1d9b700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4f40c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021195743 0 0) 0x7fd4f40cdb80 con 0x7fd4f40c8d90 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.230+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e0003570 con 0x7fd4f40c8d90 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599182533 0 0) 0x7fd4e0003570 con 0x7fd4f40c8d90 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 msgr2=0x7fd4f40c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4f40c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 >> v1:172.21.15.184:6789/0 conn(0x7fd4f415a8b0 legacy=0x7fd4f40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40cd250 con 0x7fd4f40c8d90 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fd4ec001470 con 0x7fd4f40c8d90 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 >> v1:172.21.15.98:6789/0 conn(0x7fd4f40c8d90 legacy=0x7fd4f40ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4e0003a90 0x7fd4e0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0004d50 0x7fd4e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40ccf70 con 0x7fd4e0003a90 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e000bb40 con 0x7fd4e0004d50 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e000bd00 con 0x7fd4f415ac80 2023-10-06T01:39:51.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4e0003a90 0x7fd4e0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f1d9b700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f2d9d700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0004d50 0x7fd4e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 -- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 msgr2=0x7fd4e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 -- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0004d50 msgr2=0x7fd4e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0004d50 0x7fd4e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 -- 172.21.15.98:0/3234244217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0003570 con 0x7fd4e0003a90 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f1d9b700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f415ac80 0x7fd4e0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f259c700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4e0003a90 0x7fd4e0003e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c320 tx=0x7fd4e4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7026a95eb3da0c19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4001e20 con 0x7fd4e0003a90 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400cc50 con 0x7fd4e0003a90 2023-10-06T01:39:51.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.231+0000 7fd4f159a700 1 -- 172.21.15.98:0/3234244217 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4012420 con 0x7fd4e0003a90 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 -- 172.21.15.98:0/3234244217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4e0003a90 msgr2=0x7fd4e0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/3234244217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4e0003a90 0x7fd4e0003e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c320 tx=0x7fd4e4001b10 comp rx=0 tx=0).stop 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/3234244217 reap_dead start 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 -- 172.21.15.98:0/3234244217 shutdown_connections 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 -- 172.21.15.98:0/3234244217 >> 172.21.15.98:0/3234244217 conn(0x7fd4f4067dd0 msgr2=0x7fd4f40ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 -- 172.21.15.98:0/3234244217 shutdown_connections 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.232+0000 7fd4f8b32700 1 -- 172.21.15.98:0/3234244217 wait complete. 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.233+0000 7fd4f8b32700 1 Processor -- start 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.233+0000 7fd4f8b32700 1 -- start start 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0003a90 0x7fd4f40c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f40c8770 0x7fd4f40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c6180 con 0x7fd4f4158430 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c5570 con 0x7fd4e0003a90 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c5850 con 0x7fd4f40c8770 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49122/0 (socket says 172.21.15.98:49122) 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/1022858053 learned_addr learned my addr 172.21.15.98:0/1022858053 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f1d9b700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f40c8770 0x7fd4f40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f259c700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0003a90 0x7fd4f40c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f40c8770 msgr2=0x7fd4f40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f40c8770 0x7fd4f40b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0003a90 msgr2=0x7fd4f40c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0003a90 0x7fd4f40c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 -- 172.21.15.98:0/1022858053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40c5b30 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f2d9d700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40c9360 tx=0x7fd4ec001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec010520 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec0106a0 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40c5e10 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec010a60 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.234+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f41588d0 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.235+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4064ad0 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.236+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4ec0133e0 con 0x7fd4f4158430 2023-10-06T01:39:51.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.236+0000 7fd4ea7fc700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 0x7fd4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.236+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd4ec0841b0 con 0x7fd4f4158430 2023-10-06T01:39:51.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.236+0000 7fd4f259c700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 0x7fd4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.237+0000 7fd4f259c700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 0x7fd4d40693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c320 tx=0x7fd4e4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.240+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4ec050fd0 con 0x7fd4f4158430 2023-10-06T01:39:51.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.427+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4f4158b50 con 0x7fd4f4158430 2023-10-06T01:39:51.429 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.428+0000 7fd4ea7fc700 1 -- 172.21.15.98:0/1022858053 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fd4ec0545f0 con 0x7fd4f4158430 2023-10-06T01:39:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:51.430 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 msgr2=0x7fd4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 0x7fd4d40693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c320 tx=0x7fd4e4016000 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 msgr2=0x7fd4f4155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40c9360 tx=0x7fd4ec001430 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 shutdown_connections 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd4d4066f30 0x7fd4d40693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4f4158430 0x7fd4f4155690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e0003a90 0x7fd4f40c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 --2- 172.21.15.98:0/1022858053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4f40c8770 0x7fd4f40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 >> 172.21.15.98:0/1022858053 conn(0x7fd4f4067dd0 msgr2=0x7fd4f40ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 shutdown_connections 2023-10-06T01:39:51.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.430+0000 7fd4f8b32700 1 -- 172.21.15.98:0/1022858053 wait complete. 2023-10-06T01:39:51.456 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:39:51.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.569+0000 7fb43cef4700 1 Processor -- start 2023-10-06T01:39:51.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.569+0000 7fb43cef4700 1 -- start start 2023-10-06T01:39:51.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb43cef4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4380cb630 0x7fb4380691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb43cef4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb4380ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb43cef4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb438068d50 con 0x7fb438069720 2023-10-06T01:39:51.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb43cef4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb438154060 con 0x7fb4380cb630 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb43cef4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43815a6b0 con 0x7fb43815a220 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb435d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb438069720 0x7fb438159b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37520/0 (socket says 172.21.15.98:37520) 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.570+0000 7fb435d9b700 1 -- 172.21.15.98:0/760553257 learned_addr learned my addr 172.21.15.98:0/760553257 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb436d9d700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb4380ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 --2- 172.21.15.98:0/760553257 >> v2:172.21.15.184:3300/0 conn(0x7fb4380cb630 0x7fb4380691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947573719 0 0) 0x7fb438068d50 con 0x7fb438069720 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb424003570 con 0x7fb438069720 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1904355939 0 0) 0x7fb424003570 con 0x7fb438069720 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 msgr2=0x7fb4380ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb4380ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 >> v2:172.21.15.184:3300/0 conn(0x7fb4380cb630 msgr2=0x7fb4380691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 --2- 172.21.15.98:0/760553257 >> v2:172.21.15.184:3300/0 conn(0x7fb4380cb630 0x7fb4380691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43815a990 con 0x7fb438069720 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fb420001470 con 0x7fb438069720 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 >> v1:172.21.15.98:6789/0 conn(0x7fb438069720 legacy=0x7fb438159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb424003a60 0x7fb424003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb424004d50 0x7fb4240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438154060 con 0x7fb424003a60 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43815a6b0 con 0x7fb424004d50 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42400ba60 con 0x7fb43815a220 2023-10-06T01:39:51.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb424003a60 0x7fb424003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb435d9b700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb436d9d700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb424004d50 0x7fb4240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 -- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 msgr2=0x7fb424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43815a220 0x7fb424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 -- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb424004d50 msgr2=0x7fb4240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb424004d50 0x7fb4240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 -- 172.21.15.98:0/760553257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424003570 con 0x7fb424003a60 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb436d9d700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb424004d50 0x7fb4240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.571+0000 7fb43659c700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb424003a60 0x7fb424003e70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e210 tx=0x7fb42800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555347124794ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800ea60 con 0x7fb424003a60 2023-10-06T01:39:51.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42800f040 con 0x7fb424003a60 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43559a700 1 -- 172.21.15.98:0/760553257 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800ee10 con 0x7fb424003a60 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 -- 172.21.15.98:0/760553257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb424003a60 msgr2=0x7fb424003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 --2- 172.21.15.98:0/760553257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb424003a60 0x7fb424003e70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e210 tx=0x7fb42800b040 comp rx=0 tx=0).stop 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb436d9d700 1 -- 172.21.15.98:0/760553257 reap_dead start 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 -- 172.21.15.98:0/760553257 shutdown_connections 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 -- 172.21.15.98:0/760553257 >> 172.21.15.98:0/760553257 conn(0x7fb438070810 msgr2=0x7fb438070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 -- 172.21.15.98:0/760553257 shutdown_connections 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.572+0000 7fb43cef4700 1 -- 172.21.15.98:0/760553257 wait complete. 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.573+0000 7fb43cef4700 1 Processor -- start 2023-10-06T01:39:51.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.573+0000 7fb43cef4700 1 -- start start 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb424003a60 0x7fb43816afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 0x7fb4380c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438153540 con 0x7fb438169550 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c4d60 con 0x7fb4380c1d70 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43cef4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380b2810 con 0x7fb424003a60 2023-10-06T01:39:51.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49144/0 (socket says 172.21.15.98:49144) 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 -- 172.21.15.98:0/479946275 learned_addr learned my addr 172.21.15.98:0/479946275 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb43659c700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb424003a60 0x7fb43816afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb436d9d700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 0x7fb4380c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 -- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb424003a60 msgr2=0x7fb43816afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb424003a60 0x7fb43816afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 -- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 msgr2=0x7fb4380c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 0x7fb4380c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb435d9b700 1 -- 172.21.15.98:0/479946275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4380c2700 con 0x7fb438169550 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.574+0000 7fb436d9d700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 0x7fb4380c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:51.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb435d9b700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4200074e0 tx=0x7fb420001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000fa50 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000fbd0 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4380c4a80 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200133b0 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4380b24a0 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.575+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438064ad0 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.576+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb420013720 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.576+0000 7fb42e7fc700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.576+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb420084a20 con 0x7fb438169550 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.576+0000 7fb43659c700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.577+0000 7fb43659c700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 0x7fb4180693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb42800afd0 tx=0x7fb42801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.580+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb420051840 con 0x7fb438169550 2023-10-06T01:39:51.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.806+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4380b2950 con 0x7fb438169550 2023-10-06T01:39:51.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.807+0000 7fb42e7fc700 1 -- 172.21.15.98:0/479946275 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fb420054e60 con 0x7fb438169550 2023-10-06T01:39:51.808 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:51.809 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:50.365301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T01:39:51.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 msgr2=0x7fb4180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 0x7fb4180693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb42800afd0 tx=0x7fb42801a000 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 msgr2=0x7fb4381699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4200074e0 tx=0x7fb420001e90 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 shutdown_connections 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4380c1d70 0x7fb4380c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb424003a60 0x7fb43816afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 --2- 172.21.15.98:0/479946275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb438169550 0x7fb4381699c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 >> 172.21.15.98:0/479946275 conn(0x7fb438070810 msgr2=0x7fb43815c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 shutdown_connections 2023-10-06T01:39:51.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.809+0000 7fb43cef4700 1 -- 172.21.15.98:0/479946275 wait complete. 2023-10-06T01:39:51.812 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2023-10-06T01:39:51.822 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:51.937 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.936+0000 7ff67a7c7700 1 Processor -- start 2023-10-06T01:39:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.936+0000 7ff67a7c7700 1 -- start start 2023-10-06T01:39:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff67a7c7700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff6740cbc70 0x7ff6740cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff67a7c7700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740cb7e0 con 0x7ff6740b2140 2023-10-06T01:39:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff67a7c7700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740b08a0 con 0x7ff674159480 2023-10-06T01:39:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff67a7c7700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740cb190 con 0x7ff6740cbc70 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff679fc6700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff6740b2140 0x7ff6740b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37524/0 (socket says 172.21.15.98:37524) 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff679fc6700 1 -- 172.21.15.98:0/3288997225 learned_addr learned my addr 172.21.15.98:0/3288997225 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff678fc4700 1 --2- 172.21.15.98:0/3288997225 >> v2:172.21.15.184:3300/0 conn(0x7ff6740cbc70 0x7ff6740cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24119361 0 0) 0x7ff6740cb7e0 con 0x7ff6740b2140 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff660003570 con 0x7ff6740b2140 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527529268 0 0) 0x7ff660003570 con 0x7ff6740b2140 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 >> v2:172.21.15.184:3300/0 conn(0x7ff6740cbc70 msgr2=0x7ff6740cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 --2- 172.21.15.98:0/3288997225 >> v2:172.21.15.184:3300/0 conn(0x7ff6740cbc70 0x7ff6740cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 >> v1:172.21.15.184:6789/0 conn(0x7ff674159480 legacy=0x7ff6740b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740cb470 con 0x7ff6740b2140 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.937+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7ff66c0017f0 con 0x7ff6740b2140 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 >> v1:172.21.15.98:6789/0 conn(0x7ff6740b2140 legacy=0x7ff6740b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660003a60 0x7ff660003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660004810 0x7ff660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 0x7ff66000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740cb190 con 0x7ff6600051c0 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66000c0d0 con 0x7ff660004810 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66000c290 con 0x7ff660003a60 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 0x7ff66000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff678fc4700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660004810 0x7ff660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff6797c5700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660003a60 0x7ff660003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 -- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660003a60 msgr2=0x7ff660003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660003a60 0x7ff660003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 -- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660004810 msgr2=0x7ff660004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660004810 0x7ff660004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 -- 172.21.15.98:0/3288997225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660003570 con 0x7ff6600051c0 2023-10-06T01:39:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 -- 172.21.15.98:0/3288997225 reap_dead start 2023-10-06T01:39:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 0x7ff66000ba60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00a930 tx=0x7ff66c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8ede988735b7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0070d0 con 0x7ff6600051c0 2023-10-06T01:39:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c007230 con 0x7ff6600051c0 2023-10-06T01:39:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.938+0000 7ff66bfff700 1 -- 172.21.15.98:0/3288997225 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0075d0 con 0x7ff6600051c0 2023-10-06T01:39:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 msgr2=0x7ff66000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 0x7ff66000ba60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00a930 tx=0x7ff66c0021d0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3288997225 shutdown_connections 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3288997225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600051c0 0x7ff66000ba60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3288997225 >> 172.21.15.98:0/3288997225 conn(0x7ff67406ed20 msgr2=0x7ff67406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3288997225 shutdown_connections 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.939+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3288997225 wait complete. 2023-10-06T01:39:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.940+0000 7ff67a7c7700 1 Processor -- start 2023-10-06T01:39:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.940+0000 7ff67a7c7700 1 -- start start 2023-10-06T01:39:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6600051c0 0x7ff674169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6740c8720 0x7ff6740c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741695d0 con 0x7ff6740c4170 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c90d0 con 0x7ff6600051c0 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67416a230 con 0x7ff6740c8720 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49160/0 (socket says 172.21.15.98:49160) 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 -- 172.21.15.98:0/3629612635 learned_addr learned my addr 172.21.15.98:0/3629612635 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff678fc4700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6740c8720 0x7ff6740c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff6797c5700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6600051c0 0x7ff674169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 -- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6740c8720 msgr2=0x7ff6740c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6740c8720 0x7ff6740c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 -- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6600051c0 msgr2=0x7ff674169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6600051c0 0x7ff674169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 -- 172.21.15.98:0/3629612635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67416a3f0 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff679fc6700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6740c3ad0 tx=0x7ff66c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c007b50 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c0073a0 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740c4b20 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c01d420 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.941+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740c5080 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.942+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67405f270 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.943+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff66c01d7d0 con 0x7ff6740c4170 2023-10-06T01:39:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.943+0000 7ff6697fa700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 0x7ff6580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:51.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.943+0000 7ff6797c5700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 0x7ff6580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:51.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.943+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff66c086a20 con 0x7ff6740c4170 2023-10-06T01:39:51.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.944+0000 7ff6797c5700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 0x7ff6580693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff670000c00 tx=0x7ff670007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:51.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:51.948+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff66c0537a0 con 0x7ff6740c4170 2023-10-06T01:39:52.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.133+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff674159850 con 0x7ff6740c4170 2023-10-06T01:39:52.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.134+0000 7ff6697fa700 1 -- 172.21.15.98:0/3629612635 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7ff66c056dc0 con 0x7ff6740c4170 2023-10-06T01:39:52.136 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:52.136 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:52.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 msgr2=0x7ff6580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 0x7ff6580693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff670000c00 tx=0x7ff670007d30 comp rx=0 tx=0).stop 2023-10-06T01:39:52.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 msgr2=0x7ff6740c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6740c3ad0 tx=0x7ff66c016000 comp rx=0 tx=0).stop 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 shutdown_connections 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff658066f30 0x7ff6580693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6740c4170 0x7ff6740c45e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6600051c0 0x7ff674169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 --2- 172.21.15.98:0/3629612635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6740c8720 0x7ff6740c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 >> 172.21.15.98:0/3629612635 conn(0x7ff67406ed20 msgr2=0x7ff6740b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.136+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 shutdown_connections 2023-10-06T01:39:52.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.137+0000 7ff67a7c7700 1 -- 172.21.15.98:0/3629612635 wait complete. 2023-10-06T01:39:52.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:39:44.696941+0000', 'modified': '2023-10-06T01:39:50.365301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/1024246488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6835', 'nonce': 1024246488}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1024246488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-06T01:39:52.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T01:39:52.553 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T01:39:52.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.667+0000 7f3ce1dfa700 1 Processor -- start 2023-10-06T01:39:52.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.667+0000 7f3ce1dfa700 1 -- start start 2023-10-06T01:39:52.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3ce1dfa700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3cdc1595a0 0x7f3cdc0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3ce1dfa700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3cdc0b07a0 0x7f3cdc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3ce1dfa700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc0b0ca0 con 0x7f3cdc0b49e0 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3ce1dfa700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0b0f80 con 0x7f3cdc1595a0 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3ce1dfa700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0b1260 con 0x7f3cdc0b07a0 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cdb7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3cdc1595a0 0x7f3cdc0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cd3fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3cdc0b49e0 0x7f3cdc0aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36350/0 (socket says 172.21.15.98:36350) 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cd3fff700 1 -- 172.21.15.98:0/2024289630 learned_addr learned my addr 172.21.15.98:0/2024289630 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/2024289630 >> v2:172.21.15.184:3300/0 conn(0x7f3cdc0b07a0 msgr2=0x7f3cdc0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2024289630 >> v2:172.21.15.184:3300/0 conn(0x7f3cdc0b07a0 0x7f3cdc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/2024289630 >> v1:172.21.15.98:6789/0 conn(0x7f3cdc0b49e0 legacy=0x7f3cdc0aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/2024289630 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0b1540 con 0x7f3cdc1595a0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.668+0000 7f3cdaffd700 1 -- 172.21.15.98:0/2024289630 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760145686 0 0) 0x7f3cdc0b0ca0 con 0x7f3cdc0b49e0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2024289630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0aa570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0098d0 tx=0x7f3ccc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccfe18614811eb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3cdaffd700 1 -- 172.21.15.98:0/2024289630 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc009e70 con 0x7f3cdc1595a0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3cdaffd700 1 -- 172.21.15.98:0/2024289630 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc005c10 con 0x7f3cdc1595a0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3cdaffd700 1 -- 172.21.15.98:0/2024289630 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00b610 con 0x7f3cdc1595a0 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/2024289630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 msgr2=0x7f3cdc0aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/2024289630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0aa570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0098d0 tx=0x7f3ccc009120 comp rx=0 tx=0).stop 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/2024289630 shutdown_connections 2023-10-06T01:39:52.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/2024289630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0aa570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/2024289630 >> v2:172.21.15.184:3300/0 conn(0x7f3cdc0b07a0 0x7f3cdc0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/2024289630 >> 172.21.15.98:0/2024289630 conn(0x7f3cdc099830 msgr2=0x7f3cdc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/2024289630 shutdown_connections 2023-10-06T01:39:52.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.669+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/2024289630 wait complete. 2023-10-06T01:39:52.676 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.670+0000 7f3ce1dfa700 1 Processor -- start 2023-10-06T01:39:52.676 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 -- start start 2023-10-06T01:39:52.676 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cdc0b07a0 0x7f3cdc0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cdc0b49e0 0x7f3cdc1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0aee40 con 0x7f3cdc1595a0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc157d80 con 0x7f3cdc0b07a0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3ce1dfa700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0ab780 con 0x7f3cdc0b49e0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3cdbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3cdbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33916/0 (socket says 172.21.15.98:33916) 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3cdbfff700 1 -- 172.21.15.98:0/3871864369 learned_addr learned my addr 172.21.15.98:0/3871864369 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cdc0b07a0 0x7f3cdc0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.671+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cdc0b49e0 0x7f3cdc1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 -- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cdc0b49e0 msgr2=0x7f3cdc1572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cdc0b49e0 0x7f3cdc1572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 -- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cdc0b07a0 msgr2=0x7f3cdc0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cdc0b07a0 0x7f3cdc0ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 -- 172.21.15.98:0/3871864369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0093c0 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400bc30 tx=0x7f3cd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cd37fe700 1 -- 172.21.15.98:0/3871864369 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40092e0 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cd37fe700 1 -- 172.21.15.98:0/3871864369 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd400c7a0 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc157800 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3cd37fe700 1 -- 172.21.15.98:0/3871864369 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400cb80 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.672+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc16d560 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.673+0000 7f3cd37fe700 1 -- 172.21.15.98:0/3871864369 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cd4012070 con 0x7f3cdc1595a0 2023-10-06T01:39:52.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.673+0000 7f3cd37fe700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 0x7f3cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.674+0000 7f3cd37fe700 1 -- 172.21.15.98:0/3871864369 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3cd4084620 con 0x7f3cdc1595a0 2023-10-06T01:39:52.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.674+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 0x7f3cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.674+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 0x7f3cbc0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009160 tx=0x7f3ccc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.675+0000 7f3ceb0f2b80 1 Processor -- start 2023-10-06T01:39:52.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 -- start start 2023-10-06T01:39:52.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e23c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3d7aa70 con 0x5594a3e73c00 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3d78270 con 0x5594a3e23840 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a3d74060 con 0x5594a3e24cc0 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3cdbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3cdbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33918/0 (socket says 172.21.15.98:33918) 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.676+0000 7f3cdbfff700 1 -- 172.21.15.98:0/971478315 learned_addr learned my addr 172.21.15.98:0/971478315 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e23c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 -- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 msgr2=0x5594a3e736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 -- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 msgr2=0x5594a3e23c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e23c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 -- 172.21.15.98:0/971478315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a3c0e270 con 0x5594a3e73c00 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40826f0 tx=0x7f3cd4082650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654724edae77e548 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cd0ff9700 1 -- 172.21.15.98:0/971478315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4055100 con 0x5594a3e73c00 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cd0ff9700 1 -- 172.21.15.98:0/971478315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4058350 con 0x5594a3e73c00 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3cd0ff9700 1 -- 172.21.15.98:0/971478315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd405c0a0 con 0x5594a3e73c00 2023-10-06T01:39:52.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/971478315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 msgr2=0x5594a3e74050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.677+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40826f0 tx=0x7f3cd4082650 comp rx=0 tx=0).stop 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/971478315 shutdown_connections 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e73c00 0x5594a3e74050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e23c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/971478315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/971478315 >> 172.21.15.98:0/971478315 conn(0x5594a3e25160 msgr2=0x5594a3e25530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/971478315 shutdown_connections 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/971478315 wait complete. 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 -- start start 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.678+0000 7f3ceb0f2b80 1 Processor -- start 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e64620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e73c00 0x5594a3e48bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3d1f890 con 0x5594a3e23840 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3d3aea0 con 0x5594a3e73c00 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a3bf6e30 con 0x5594a3e24cc0 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3ceb0f2b80 10 client.0 _ll_register_callbacks cb 0x5594a3de6e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33920/0 (socket says 172.21.15.98:33920) 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1633605297 learned_addr learned my addr 172.21.15.98:0/1633605297 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e64620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e73c00 0x5594a3e48bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 msgr2=0x5594a3e64620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e64620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e73c00 msgr2=0x5594a3e48bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e73c00 0x5594a3e48bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1633605297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc16d560 con 0x5594a3e23840 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006d60 tx=0x7f3ccc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a854e2b218aa0ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.679+0000 7f3cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc02f070 con 0x5594a3e23840 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3ccc006390 con 0x5594a3e23840 2023-10-06T01:39:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594a3e18940 con 0x5594a3e23840 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ccc022070 con 0x5594a3e23840 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 10 client.4461 fetch_fsmap learned FSMap version 6 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5594a3bb0fe0 con 0x5594a3e23840 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594a3e18940 con 0x5594a3e23840 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f3ccc006760 con 0x5594a3e23840 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 10 client.4461 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 10 client.4461 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 20 client.4461 populate_metadata read hostname 'smithi098' 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e6e340 0x5594a3e6e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 4 client.4461 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T01:39:52.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 --> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594a3df5870 con 0x5594a3e6e340 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e6e340 0x5594a3e6e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cb9ffb700 10 client.4461 ms_handle_connect on v2:172.21.15.98:6835/1024246488 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.680+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e6e340 0x5594a3e6e710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990e87882d906f80 server_cookie=425971f8732e2457 in_seq=0 out_seq=0 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/1633605297 <== mds.0 v2:172.21.15.98:6835/1024246488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5594a3df5870 con 0x5594a3e6e340 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3cb9ffb700 10 client.4461 handle_command_reply: tid=0 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 client.4461 shutdown 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 msgr2=0x5594a3e472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006d60 tx=0x7f3ccc022000 comp rx=0 tx=0).stop 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 shutdown_connections 2023-10-06T01:39:52.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e73c00 0x5594a3e48bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e472e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e6e340 0x5594a3e6e710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1633605297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e24cc0 0x5594a3e64620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3cb9ffb700 0 client.4461 ms_handle_reset on v2:172.21.15.98:6835/1024246488 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 >> 172.21.15.98:0/1633605297 conn(0x5594a3e25160 msgr2=0x5594a3e241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 shutdown_connections 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1633605297 wait complete. 2023-10-06T01:39:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.681+0000 7f3ceb0f2b80 20 client.4461 trim_cache size 0 max 16384 2023-10-06T01:39:52.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.698+0000 7f3ceb0f2b80 1 Processor -- start 2023-10-06T01:39:52.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.698+0000 7f3ceb0f2b80 1 -- start start 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e24cc0 0x5594a3e48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e8b760 con 0x5594a3e23840 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e1ea10 con 0x5594a3e24cc0 2023-10-06T01:39:52.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e1e620 con 0x5594a3e73c00 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33922/0 (socket says 172.21.15.98:33922) 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1004033231 learned_addr learned my addr 172.21.15.98:0/1004033231 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e24cc0 0x5594a3e48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 msgr2=0x5594a3e7c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e7c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e24cc0 msgr2=0x5594a3e48c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e24cc0 0x5594a3e48c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 -- 172.21.15.98:0/1004033231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a3e4d1a0 con 0x5594a3e23840 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006100 tx=0x7f3ccc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49ff8ad12823611 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cd17fa700 1 -- 172.21.15.98:0/1004033231 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc022040 con 0x5594a3e23840 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cd17fa700 1 -- 172.21.15.98:0/1004033231 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc006390 con 0x5594a3e23840 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.699+0000 7f3cd17fa700 1 -- 172.21.15.98:0/1004033231 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc034be0 con 0x5594a3e23840 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1004033231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 msgr2=0x5594a3e6e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006100 tx=0x7f3ccc02f000 comp rx=0 tx=0).stop 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1004033231 shutdown_connections 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e23840 0x5594a3e6e9f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/1004033231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e24cc0 0x5594a3e48c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1004033231 >> 172.21.15.98:0/1004033231 conn(0x5594a3e25160 msgr2=0x5594a3e241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1004033231 shutdown_connections 2023-10-06T01:39:52.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/1004033231 wait complete. 2023-10-06T01:39:52.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 -- start start 2023-10-06T01:39:52.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.700+0000 7f3ceb0f2b80 1 Processor -- start 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e6e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e4a800 con 0x5594a3e24cc0 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e20d90 con 0x5594a3e23840 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a3e1ef30 con 0x5594a3e73c00 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3ceb0f2b80 10 client.0 _ll_register_callbacks cb 0x5594a3de6e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:39:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33936/0 (socket says 172.21.15.98:33936) 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 -- 172.21.15.98:0/2585199501 learned_addr learned my addr 172.21.15.98:0/2585199501 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e6e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cdbfff700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 -- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 msgr2=0x5594a3e72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e72d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.701+0000 7f3cd3fff700 1 -- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 msgr2=0x5594a3e6e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e6e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cd3fff700 1 -- 172.21.15.98:0/2585199501 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a3bb0fe0 con 0x5594a3e24cc0 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e6e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cd3fff700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc099f60 tx=0x7f3cc4006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51927d4e8a78be76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc401f070 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3cc4021d80 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd4058350 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc4006250 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 10 client.4467 fetch_fsmap learned FSMap version 6 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5594a3d42780 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4058350 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f3cc40063b0 con 0x5594a3e24cc0 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 10 client.4467 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T01:39:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 10 client.4467 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 20 client.4467 populate_metadata read hostname 'smithi098' 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e7c820 0x5594a3e7cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 4 client.4467 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 --> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5594a3d16e00 con 0x5594a3e7c820 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e7c820 0x5594a3e7cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cdb7fe700 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e7c820 0x5594a3e7cbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a82643462f9381d server_cookie=c23ddfd94f721775 in_seq=0 out_seq=0 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.702+0000 7f3cb9ffb700 10 client.4467 ms_handle_connect on v2:172.21.15.98:6835/1024246488 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3cb9ffb700 1 -- 172.21.15.98:0/2585199501 <== mds.0 v2:172.21.15.98:6835/1024246488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x5594a3d16e00 con 0x5594a3e7c820 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3cb9ffb700 10 client.4467 handle_command_reply: tid=0 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 client.4467 shutdown 2023-10-06T01:39:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 msgr2=0x5594a3e49d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc099f60 tx=0x7f3cc4006c50 comp rx=0 tx=0).stop 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 shutdown_connections 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5594a3e7c820 0x5594a3e7cbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5594a3e73c00 0x5594a3e72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594a3e23840 0x5594a3e6e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3cb9ffb700 0 client.4467 ms_handle_reset on v2:172.21.15.98:6835/1024246488 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 --2- 172.21.15.98:0/2585199501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594a3e24cc0 0x5594a3e49d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 >> 172.21.15.98:0/2585199501 conn(0x5594a3e25160 msgr2=0x5594a3e241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 shutdown_connections 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 1 -- 172.21.15.98:0/2585199501 wait complete. 2023-10-06T01:39:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ceb0f2b80 20 client.4467 trim_cache size 0 max 16384 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "c6dd341a-820f-4e37-8897-3250639f93a1", 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4296, 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2023-10-06T01:39:52.714 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 7.395256056, 2023-10-06T01:39:52.715 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 6, 2023-10-06T01:39:52.715 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 24, 2023-10-06T01:39:52.715 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T01:39:52.715 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 13.312268693 2023-10-06T01:39:52.715 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:39:52.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 msgr2=0x7f3cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 0x7f3cbc0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009160 tx=0x7f3ccc016000 comp rx=0 tx=0).stop 2023-10-06T01:39:52.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 msgr2=0x7f3cdc0ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400bc30 tx=0x7f3cd4009840 comp rx=0 tx=0).stop 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 shutdown_connections 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cbc066f30 0x7f3cbc0693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cdc1595a0 0x7f3cdc0ad9c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cdc0b07a0 0x7f3cdc0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 --2- 172.21.15.98:0/3871864369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cdc0b49e0 0x7f3cdc1572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.703+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 >> 172.21.15.98:0/3871864369 conn(0x7f3cdc099830 msgr2=0x7f3cdc0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.704+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 shutdown_connections 2023-10-06T01:39:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.704+0000 7f3ce1dfa700 1 -- 172.21.15.98:0/3871864369 wait complete. 2023-10-06T01:39:52.718 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:39:52.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.829+0000 7f1663413700 1 Processor -- start 2023-10-06T01:39:52.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.829+0000 7f1663413700 1 -- start start 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.830+0000 7f1663413700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.830+0000 7f1663413700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.830+0000 7f1663413700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165c06c9e0 con 0x7f165c156390 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.830+0000 7f1663413700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165c156b60 con 0x7f165c0cc920 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.830+0000 7f1663413700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c156e40 con 0x7f165c06c1a0 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16609ae700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16609ae700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49622/0 (socket says 172.21.15.98:49622) 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16611af700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f165c156390 0x7f165c0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35714/0 (socket says 172.21.15.98:35714) 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58550/0 (socket says 172.21.15.98:58550) 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16609ae700 1 -- 172.21.15.98:0/2812931266 learned_addr learned my addr 172.21.15.98:0/2812931266 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 -- 172.21.15.98:0/2812931266 >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 msgr2=0x7f165c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 --2- 172.21.15.98:0/2812931266 >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 -- 172.21.15.98:0/2812931266 >> v1:172.21.15.184:6789/0 conn(0x7f165c156390 legacy=0x7f165c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 -- 172.21.15.98:0/2812931266 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c157120 con 0x7f165c06c1a0 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16609ae700 1 --2- 172.21.15.98:0/2812931266 >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f16619b0700 1 --2- 172.21.15.98:0/2812931266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f16540098e0 tx=0x7f16540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1decec74637c5786 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f165bfff700 1 -- 172.21.15.98:0/2812931266 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400b640 con 0x7f165c06c1a0 2023-10-06T01:39:52.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f165bfff700 1 -- 172.21.15.98:0/2812931266 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165400b7a0 con 0x7f165c06c1a0 2023-10-06T01:39:52.835 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.831+0000 7f165bfff700 1 -- 172.21.15.98:0/2812931266 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400bb40 con 0x7f165c06c1a0 2023-10-06T01:39:52.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 -- 172.21.15.98:0/2812931266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 msgr2=0x7f165c0c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 --2- 172.21.15.98:0/2812931266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f16540098e0 tx=0x7f16540091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 -- 172.21.15.98:0/2812931266 shutdown_connections 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 --2- 172.21.15.98:0/2812931266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c06c1a0 0x7f165c0c88c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 --2- 172.21.15.98:0/2812931266 >> v2:172.21.15.184:3300/0 conn(0x7f165c0cc920 0x7f165c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 -- 172.21.15.98:0/2812931266 >> 172.21.15.98:0/2812931266 conn(0x7f165c0cdef0 msgr2=0x7f165c0c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 -- 172.21.15.98:0/2812931266 shutdown_connections 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.832+0000 7f1663413700 1 -- 172.21.15.98:0/2812931266 wait complete. 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.833+0000 7f1663413700 1 Processor -- start 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.833+0000 7f1663413700 1 -- start start 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f165c06c1a0 0x7f165c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c156390 0x7f165c0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c16b010 con 0x7f165c0cc920 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c152230 con 0x7f165c06c1a0 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f1663413700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c0b3040 con 0x7f165c156390 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33944/0 (socket says 172.21.15.98:33944) 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 -- 172.21.15.98:0/3250367120 learned_addr learned my addr 172.21.15.98:0/3250367120 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16619b0700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c156390 0x7f165c0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16611af700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f165c06c1a0 0x7f165c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 -- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c156390 msgr2=0x7f165c0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c156390 0x7f165c0c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 -- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f165c06c1a0 msgr2=0x7f165c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f165c06c1a0 0x7f165c151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 -- 172.21.15.98:0/3250367120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500094f0 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16609ae700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f165000ec80 tx=0x7f1650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.834+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165000c850 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.835+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165000c9b0 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.835+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1654009420 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.835+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165000cd30 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.835+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165c0b2d60 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.835+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165c064ad0 con 0x7f165c0cc920 2023-10-06T01:39:52.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.836+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1650013680 con 0x7f165c0cc920 2023-10-06T01:39:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.836+0000 7f16597fa700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 0x7f16480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.836+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1650084f80 con 0x7f165c0cc920 2023-10-06T01:39:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.836+0000 7f16611af700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 0x7f16480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.837+0000 7f16611af700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 0x7f16480693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f165c0c2b70 tx=0x7f164c001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:52.840+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1650051da0 con 0x7f165c0cc920 2023-10-06T01:39:53.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.066+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f165c16b150 con 0x7f165c0cc920 2023-10-06T01:39:53.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.066+0000 7f16597fa700 1 -- 172.21.15.98:0/3250367120 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1650017070 con 0x7f165c0cc920 2023-10-06T01:39:53.068 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T01:39:53.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.068+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 msgr2=0x7f16480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.068+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 0x7f16480693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f165c0c2b70 tx=0x7f164c001a30 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.068+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 msgr2=0x7f165c0aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.068+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f165000ec80 tx=0x7f1650009250 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 shutdown_connections 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1648066f30 0x7f16480693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f165c156390 0x7f165c0c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f165c06c1a0 0x7f165c151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 --2- 172.21.15.98:0/3250367120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f165c0cc920 0x7f165c0aec90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 >> 172.21.15.98:0/3250367120 conn(0x7f165c0cdef0 msgr2=0x7f165c155230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:53.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 shutdown_connections 2023-10-06T01:39:53.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:53.069+0000 7f1663413700 1 -- 172.21.15.98:0/3250367120 wait complete. 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T01:39:53.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:39:53.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T01:39:53.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.083 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2023-10-06T01:39:53.098 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:39:53.098 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:39:53.098 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:39:53.098 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:39:53.098 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T01:39:53.099 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:39:53.100 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41945sec preferred_lft 41945sec 2023-10-06T01:39:53.100 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2023-10-06T01:39:53.100 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:39:53.100 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T01:39:53.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T01:39:53.101 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:39:53.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.257 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:39:53.297 INFO:teuthology.orchestra.run.smithi098.stdout:1 2023-10-06T01:39:53.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.298 DEBUG:teuthology.orchestra.run.smithi098:> route 2023-10-06T01:39:53.374 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2023-10-06T01:39:53.374 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:39:53.374 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:39:53.374 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:39:53.374 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:39:53.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:39:53.375 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:39:53.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.659 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:39:53.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.674 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2023-10-06T01:39:53.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.729 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:39:53.729 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:39:53.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T01:39:53.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-06T01:39:53.730 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:39:53.819 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-06T01:39:53.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:53.819 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:39:53.819 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:39:53.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-06T01:39:53.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T01:39:53.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-06T01:39:53.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-06T01:39:53.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-06T01:39:53.820 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:39:54.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:54.016 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:39:54.016 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:39:54.016 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2023-10-06T01:39:54.016 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T01:39:54.016 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:39:54.108 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-06T01:39:54.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:39:54.109 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:54.123 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T01:39:54.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:39:54.123 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:54.178 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2023-10-06T01:39:54.243 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T01:39:54.243 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T01:39:54.307 INFO:teuthology.orchestra.run.smithi098.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T01:39:54.307 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T01:39:54.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:54.308 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2023-10-06T01:39:54.363 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T01:39:54.363 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-06T01:39:54.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2023-10-06T01:39:54.429 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T01:39:54.429 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T01:39:54.494 INFO:teuthology.orchestra.run.smithi098.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T01:39:54.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:2023-10-06T01:39:54.494+0000 7f85fa9f9540 -1 init, newargv = 0x55ca2d81ae10 newargc=15 2023-10-06T01:39:54.495 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T01:39:54.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:54.496 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2023-10-06T01:39:54.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[128463]: starting ceph client 2023-10-06T01:39:54.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[128463]: starting fuse 2023-10-06T01:39:54.514 INFO:teuthology.orchestra.run.smithi098.stdout:46 2023-10-06T01:39:54.514 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> import re 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> import os 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-06T01:39:54.515 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2023-10-06T01:39:54.516 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> return f 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.admin")) 2023-10-06T01:39:54.517 DEBUG:teuthology.orchestra.run.smithi098:> ' 2023-10-06T01:39:54.635 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.admin.128463.asok 2023-10-06T01:39:54.817 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.128463.asok 2023-10-06T01:39:54.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:54.818 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.128463.asok status 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "admin", 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2023-10-06T01:39:54.968 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "128463", 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4476, 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 4476 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:39:54.969 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1050676503 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1050676503 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.4476 192.168.0.1:0/1050676503", 2023-10-06T01:39:54.970 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1050676503", 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 6, 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 24, 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2023-10-06T01:39:54.971 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:39:54.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:54.977 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2023-10-06T01:39:54.978 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:54.992 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2023-10-06T01:39:54.993 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:54.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:54.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:55.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:39:55.058 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-06T01:39:55.164 INFO:teuthology.orchestra.run.smithi098.stdout:# file: . 2023-10-06T01:39:55.164 INFO:teuthology.orchestra.run.smithi098.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-06T01:39:55.164 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:55.164 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2023-10-06T01:39:55.164 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:39:55.165 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-06T01:39:55.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[128463]: fuse finished with error 0 and tester_r 0 2023-10-06T01:39:55.199 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T01:39:55.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:39:55.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:39:55.285 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-06T01:39:55.300 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:55.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.454+0000 7f2d57982700 1 Processor -- start 2023-10-06T01:39:55.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.454+0000 7f2d57982700 1 -- start start 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d501595a0 0x7f2d500aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d500b49e0 0x7f2d500aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d500b07a0 0x7f2d500b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500b0ca0 con 0x7f2d500b07a0 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500b0f80 con 0x7f2d500b49e0 2023-10-06T01:39:55.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d57982700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500b1260 con 0x7f2d501595a0 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d500b07a0 0x7f2d500b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d500b07a0 0x7f2d500b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33946/0 (socket says 172.21.15.98:33946) 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 -- 172.21.15.98:0/727320718 learned_addr learned my addr 172.21.15.98:0/727320718 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d54f1d700 1 --2- 172.21.15.98:0/727320718 >> v2:172.21.15.184:3300/0 conn(0x7f2d500b49e0 0x7f2d500aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d5571e700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d501595a0 0x7f2d500aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 -- 172.21.15.98:0/727320718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d501595a0 msgr2=0x7f2d500aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d501595a0 0x7f2d500aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 -- 172.21.15.98:0/727320718 >> v2:172.21.15.184:3300/0 conn(0x7f2d500b49e0 msgr2=0x7f2d500aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/727320718 >> v2:172.21.15.184:3300/0 conn(0x7f2d500b49e0 0x7f2d500aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 -- 172.21.15.98:0/727320718 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500b1540 con 0x7f2d500b07a0 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d500b07a0 0x7f2d500b38d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d50089060 tx=0x7f2d4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00022e351e6efed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d47fff700 1 -- 172.21.15.98:0/727320718 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00b640 con 0x7f2d500b07a0 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d47fff700 1 -- 172.21.15.98:0/727320718 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c00b7a0 con 0x7f2d500b07a0 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.455+0000 7f2d47fff700 1 -- 172.21.15.98:0/727320718 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00bb60 con 0x7f2d500b07a0 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 -- 172.21.15.98:0/727320718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d500b07a0 msgr2=0x7f2d500b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d500b07a0 0x7f2d500b38d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d50089060 tx=0x7f2d4c009250 comp rx=0 tx=0).stop 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 -- 172.21.15.98:0/727320718 shutdown_connections 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d500b07a0 0x7f2d500b38d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 --2- 172.21.15.98:0/727320718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d501595a0 0x7f2d500aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 --2- 172.21.15.98:0/727320718 >> v2:172.21.15.184:3300/0 conn(0x7f2d500b49e0 0x7f2d500aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 -- 172.21.15.98:0/727320718 >> 172.21.15.98:0/727320718 conn(0x7f2d50099830 msgr2=0x7f2d5006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:55.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 -- 172.21.15.98:0/727320718 shutdown_connections 2023-10-06T01:39:55.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.456+0000 7f2d57982700 1 -- 172.21.15.98:0/727320718 wait complete. 2023-10-06T01:39:55.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.457+0000 7f2d57982700 1 Processor -- start 2023-10-06T01:39:55.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.457+0000 7f2d57982700 1 -- start start 2023-10-06T01:39:55.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d500b07a0 0x7f2d50157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d500b49e0 0x7f2d500c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c7260 con 0x7f2d501595a0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501580c0 con 0x7f2d500b07a0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d57982700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c5b70 con 0x7f2d500b49e0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d55f1f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d55f1f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33960/0 (socket says 172.21.15.98:33960) 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d55f1f700 1 -- 172.21.15.98:0/1375771906 learned_addr learned my addr 172.21.15.98:0/1375771906 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d5571e700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d500b07a0 0x7f2d50157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d54f1d700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d500b49e0 0x7f2d500c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d55f1f700 1 -- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d500b49e0 msgr2=0x7f2d500c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.458+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d500b49e0 0x7f2d500c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d55f1f700 1 -- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d500b07a0 msgr2=0x7f2d50157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d500b07a0 0x7f2d50157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d55f1f700 1 -- 172.21.15.98:0/1375771906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0094f0 con 0x7f2d501595a0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d55f1f700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0094c0 tx=0x7f2d4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00bb20 con 0x7f2d501595a0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c011420 con 0x7f2d501595a0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d500c5e50 con 0x7f2d501595a0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0117e0 con 0x7f2d501595a0 2023-10-06T01:39:55.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.459+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d500c6ef0 con 0x7f2d501595a0 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.460+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d500c73d0 con 0x7f2d501595a0 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.460+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d4c011b50 con 0x7f2d501595a0 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.460+0000 7f2d457fa700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.461+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2d4c086350 con 0x7f2d501595a0 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.461+0000 7f2d5571e700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.461+0000 7f2d5571e700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 0x7f2d340693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f510 tx=0x7f2d4800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.465+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d4c0531f0 con 0x7f2d501595a0 2023-10-06T01:39:55.650 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.648+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d50064ad0 con 0x7f2d501595a0 2023-10-06T01:39:55.651 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.649+0000 7f2d457fa700 1 -- 172.21.15.98:0/1375771906 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f2d4c056810 con 0x7f2d501595a0 2023-10-06T01:39:55.651 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:55.651 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:55.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 msgr2=0x7f2d340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 0x7f2d340693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f510 tx=0x7f2d4800b040 comp rx=0 tx=0).stop 2023-10-06T01:39:55.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 msgr2=0x7f2d500c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0094c0 tx=0x7f2d4c017000 comp rx=0 tx=0).stop 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 shutdown_connections 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d501595a0 0x7f2d500c76f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d500b07a0 0x7f2d50157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 --2- 172.21.15.98:0/1375771906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d500b49e0 0x7f2d500c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.651+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 >> 172.21.15.98:0/1375771906 conn(0x7f2d50099830 msgr2=0x7f2d50069610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.652+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 shutdown_connections 2023-10-06T01:39:55.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.652+0000 7f2d57982700 1 -- 172.21.15.98:0/1375771906 wait complete. 2023-10-06T01:39:55.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.778+0000 7f9707fff700 1 Processor -- start 2023-10-06T01:39:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.778+0000 7f9707fff700 1 -- start start 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9707fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97080c6060 0x7f97080ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9707fff700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080ae250 con 0x7f9708159580 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9707fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080ad640 con 0x7f9708159950 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9707fff700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97080ad920 con 0x7f97080c6060 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9706ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9708159580 0x7f97080ace60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36364/0 (socket says 172.21.15.98:36364) 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f97077fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97080c6060 0x7f97080ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9706ffd700 1 -- 172.21.15.98:0/363862808 learned_addr learned my addr 172.21.15.98:0/363862808 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:55.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f97077fe700 1 -- 172.21.15.98:0/363862808 >> v1:172.21.15.184:6789/0 conn(0x7f9708159950 legacy=0x7f97080ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f97077fe700 1 -- 172.21.15.98:0/363862808 >> v1:172.21.15.98:6789/0 conn(0x7f9708159580 legacy=0x7f97080ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f97077fe700 1 -- 172.21.15.98:0/363862808 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080adc00 con 0x7f97080c6060 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f9705ffb700 1 -- 172.21.15.98:0/363862808 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673030534 0 0) 0x7f97080ae250 con 0x7f9708159580 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.779+0000 7f97077fe700 1 --2- 172.21.15.98:0/363862808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080ce890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e60 tx=0x7f9700009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9c343a89e48aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9705ffb700 1 -- 172.21.15.98:0/363862808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000e070 con 0x7f97080c6060 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9705ffb700 1 -- 172.21.15.98:0/363862808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970000bc30 con 0x7f97080c6060 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9705ffb700 1 -- 172.21.15.98:0/363862808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700012430 con 0x7f97080c6060 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 -- 172.21.15.98:0/363862808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 msgr2=0x7f97080ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 --2- 172.21.15.98:0/363862808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080ce890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e60 tx=0x7f9700009780 comp rx=0 tx=0).stop 2023-10-06T01:39:55.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 -- 172.21.15.98:0/363862808 shutdown_connections 2023-10-06T01:39:55.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 --2- 172.21.15.98:0/363862808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080ce890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 -- 172.21.15.98:0/363862808 >> 172.21.15.98:0/363862808 conn(0x7f9708099560 msgr2=0x7f970806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:55.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 -- 172.21.15.98:0/363862808 shutdown_connections 2023-10-06T01:39:55.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.780+0000 7f9707fff700 1 -- 172.21.15.98:0/363862808 wait complete. 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.781+0000 7f9707fff700 1 Processor -- start 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.781+0000 7f9707fff700 1 -- start start 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 0x7f97080b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 0x7f970815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708156ec0 con 0x7f97080c6060 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970815bb80 con 0x7f9708159950 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9707fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080b05b0 con 0x7f9708159580 2023-10-06T01:39:55.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33974/0 (socket says 172.21.15.98:33974) 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 -- 172.21.15.98:0/1036004758 learned_addr learned my addr 172.21.15.98:0/1036004758 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f97077fe700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 0x7f970815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f97067fc700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 0x7f97080b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 -- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 msgr2=0x7f97080b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 0x7f97080b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 -- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 msgr2=0x7f970815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 0x7f970815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f9706ffd700 1 -- 172.21.15.98:0/1036004758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700009a20 con 0x7f97080c6060 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f97067fc700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 0x7f97080b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:55.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.782+0000 7f97077fe700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 0x7f970815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f9706ffd700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c5b0 tx=0x7f96f8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8005b30 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8005c90 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970815ac70 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8007300 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.783+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970815b5b0 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.784+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970815be60 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.784+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96f801b3b0 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.784+0000 7f96eeffd700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.785+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f96f8083e90 con 0x7f97080c6060 2023-10-06T01:39:55.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.785+0000 7f97067fc700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:55.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.785+0000 7f97067fc700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 0x7f96f40693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96fc007fd0 tx=0x7f96fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:55.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.789+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96f8050cb0 con 0x7f97080c6060 2023-10-06T01:39:55.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.972+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f970805f270 con 0x7f97080c6060 2023-10-06T01:39:55.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.973+0000 7f96eeffd700 1 -- 172.21.15.98:0/1036004758 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f96f8017020 con 0x7f97080c6060 2023-10-06T01:39:55.975 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:55.975 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:55.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 msgr2=0x7f96f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 0x7f96f40693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96fc007fd0 tx=0x7f96fc009040 comp rx=0 tx=0).stop 2023-10-06T01:39:55.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 msgr2=0x7f97080b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c5b0 tx=0x7f96f8001070 comp rx=0 tx=0).stop 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 shutdown_connections 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f96f4066f30 0x7f96f40693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9708159950 0x7f970815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080c6060 0x7f97080b5a90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 --2- 172.21.15.98:0/1036004758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9708159580 0x7f97080b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 >> 172.21.15.98:0/1036004758 conn(0x7f9708099560 msgr2=0x7f9708158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 shutdown_connections 2023-10-06T01:39:55.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:55.975+0000 7f9707fff700 1 -- 172.21.15.98:0/1036004758 wait complete. 2023-10-06T01:39:55.989 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T01:39:56.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.104+0000 7f53a97d5700 1 Processor -- start 2023-10-06T01:39:56.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.104+0000 7f53a97d5700 1 -- start start 2023-10-06T01:39:56.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.105+0000 7f53a97d5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53a40cc3d0 0x7f53a4068b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:56.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.105+0000 7f53a97d5700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a4067fc0 con 0x7f53a406e390 2023-10-06T01:39:56.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.105+0000 7f53a97d5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a4067690 con 0x7f53a4159430 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.105+0000 7f53a97d5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a406eb00 con 0x7f53a40cc3d0 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.105+0000 7f53a37fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53a40cc3d0 0x7f53a4068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a27fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f53a406e390 0x7f53a4068450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36372/0 (socket says 172.21.15.98:36372) 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a37fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53a40cc3d0 0x7f53a4068b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33988/0 (socket says 172.21.15.98:33988) 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a27fc700 1 -- 172.21.15.98:0/1893783891 learned_addr learned my addr 172.21.15.98:0/1893783891 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a37fe700 1 -- 172.21.15.98:0/1893783891 >> v1:172.21.15.184:6789/0 conn(0x7f53a4159430 legacy=0x7f53a406dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a37fe700 1 -- 172.21.15.98:0/1893783891 >> v1:172.21.15.98:6789/0 conn(0x7f53a406e390 legacy=0x7f53a4068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a37fe700 1 -- 172.21.15.98:0/1893783891 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4067970 con 0x7f53a40cc3d0 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a1ffb700 1 -- 172.21.15.98:0/1893783891 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306419059 0 0) 0x7f53a4067fc0 con 0x7f53a406e390 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a37fe700 1 --2- 172.21.15.98:0/1893783891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a40cc3d0 0x7f53a4068b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098e0 tx=0x7f539c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1826d526dd464b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a1ffb700 1 -- 172.21.15.98:0/1893783891 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c009e80 con 0x7f53a40cc3d0 2023-10-06T01:39:56.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a1ffb700 1 -- 172.21.15.98:0/1893783891 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c00b820 con 0x7f53a40cc3d0 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.106+0000 7f53a1ffb700 1 -- 172.21.15.98:0/1893783891 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00bba0 con 0x7f53a40cc3d0 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 -- 172.21.15.98:0/1893783891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a40cc3d0 msgr2=0x7f53a4068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 --2- 172.21.15.98:0/1893783891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a40cc3d0 0x7f53a4068b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098e0 tx=0x7f539c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 -- 172.21.15.98:0/1893783891 shutdown_connections 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 --2- 172.21.15.98:0/1893783891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a40cc3d0 0x7f53a4068b60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 -- 172.21.15.98:0/1893783891 >> 172.21.15.98:0/1893783891 conn(0x7f53a40cd410 msgr2=0x7f53a40b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 -- 172.21.15.98:0/1893783891 shutdown_connections 2023-10-06T01:39:56.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.107+0000 7f53a97d5700 1 -- 172.21.15.98:0/1893783891 wait complete. 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.108+0000 7f53a97d5700 1 Processor -- start 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.108+0000 7f53a97d5700 1 -- start start 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40cc3d0 0x7f53a40a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a4159430 0x7f53a415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415c860 con 0x7f53a406e390 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a416de40 con 0x7f53a40cc3d0 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415c210 con 0x7f53a4159430 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:56.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33994/0 (socket says 172.21.15.98:33994) 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 -- 172.21.15.98:0/3325043802 learned_addr learned my addr 172.21.15.98:0/3325043802 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a37fe700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a4159430 0x7f53a415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a27fc700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40cc3d0 0x7f53a40a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 -- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a4159430 msgr2=0x7f53a415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a4159430 0x7f53a415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 -- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40cc3d0 msgr2=0x7f53a40a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40cc3d0 0x7f53a40a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 -- 172.21.15.98:0/3325043802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c009420 con 0x7f53a406e390 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a2ffd700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f539400bc30 tx=0x7f5394009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400c9b0 con 0x7f53a406e390 2023-10-06T01:39:56.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400cb50 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a415c4f0 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394014410 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.109+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a40a5a10 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.110+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a415c070 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.111+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5394014800 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.111+0000 7f539affd700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 0x7f53840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.111+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5394084a70 con 0x7f53a406e390 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.111+0000 7f53a27fc700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 0x7f53840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:56.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.111+0000 7f53a27fc700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 0x7f53840693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f538c007fd0 tx=0x7f538c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:56.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.115+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53a415c070 con 0x7f53a406e390 2023-10-06T01:39:56.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:56.341+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53a415c070 con 0x7f53a406e390 2023-10-06T01:39:57.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.009+0000 7f539affd700 1 -- 172.21.15.98:0/3325043802 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f53a415c070 con 0x7f53a406e390 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 msgr2=0x7f53840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 0x7f53840693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f538c007fd0 tx=0x7f538c009040 comp rx=0 tx=0).stop 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 msgr2=0x7f53a4143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f539400bc30 tx=0x7f5394009820 comp rx=0 tx=0).stop 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 shutdown_connections 2023-10-06T01:39:57.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5384066f30 0x7f53840693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53a4159430 0x7f53a415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a406e390 0x7f53a4143700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 --2- 172.21.15.98:0/3325043802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53a40cc3d0 0x7f53a40a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 >> 172.21.15.98:0/3325043802 conn(0x7f53a40cd410 msgr2=0x7f53a4155f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 shutdown_connections 2023-10-06T01:39:57.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.011+0000 7f53a97d5700 1 -- 172.21.15.98:0/3325043802 wait complete. 2023-10-06T01:39:57.027 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T01:39:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.146+0000 7f9915252700 1 Processor -- start 2023-10-06T01:39:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.146+0000 7f9915252700 1 -- start start 2023-10-06T01:39:57.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.146+0000 7f9915252700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 0x7f99100ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.146+0000 7f9915252700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.146+0000 7f9915252700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99100cf610 con 0x7f9910159510 2023-10-06T01:39:57.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f9915252700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99100cf8f0 con 0x7f99101598e0 2023-10-06T01:39:57.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f9915252700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100cfbd0 con 0x7f99100cbcb0 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990e59c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 0x7f99100ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58606/0 (socket says 172.21.15.98:58606) 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990ed9d700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9910159510 0x7f99100cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35734/0 (socket says 172.21.15.98:35734) 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 -- 172.21.15.98:0/2545517889 learned_addr learned my addr 172.21.15.98:0/2545517889 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 -- 172.21.15.98:0/2545517889 >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 msgr2=0x7f99100ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 --2- 172.21.15.98:0/2545517889 >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 0x7f99100ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 -- 172.21.15.98:0/2545517889 >> v1:172.21.15.184:6789/0 conn(0x7f9910159510 legacy=0x7f99100cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 -- 172.21.15.98:0/2545517889 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100cfeb0 con 0x7f99100cbcb0 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990e59c700 1 --2- 172.21.15.98:0/2545517889 >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 0x7f99100ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:57.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.147+0000 7f990f59e700 1 --2- 172.21.15.98:0/2545517889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9908009f20 tx=0x7f9908009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed540369180b7ced server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:57.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f990dd9b700 1 -- 172.21.15.98:0/2545517889 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800e070 con 0x7f99100cbcb0 2023-10-06T01:39:57.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f990dd9b700 1 -- 172.21.15.98:0/2545517889 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800b9d0 con 0x7f99100cbcb0 2023-10-06T01:39:57.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f990dd9b700 1 -- 172.21.15.98:0/2545517889 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080092e0 con 0x7f99100cbcb0 2023-10-06T01:39:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 -- 172.21.15.98:0/2545517889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 msgr2=0x7f99100ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 --2- 172.21.15.98:0/2545517889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9908009f20 tx=0x7f9908009840 comp rx=0 tx=0).stop 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 -- 172.21.15.98:0/2545517889 shutdown_connections 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 --2- 172.21.15.98:0/2545517889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99100cbcb0 0x7f99100ced60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 --2- 172.21.15.98:0/2545517889 >> v2:172.21.15.184:3300/0 conn(0x7f99101598e0 0x7f99100ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 -- 172.21.15.98:0/2545517889 >> 172.21.15.98:0/2545517889 conn(0x7f9910089e90 msgr2=0x7f99100997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 -- 172.21.15.98:0/2545517889 shutdown_connections 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.148+0000 7f9915252700 1 -- 172.21.15.98:0/2545517889 wait complete. 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.149+0000 7f9915252700 1 Processor -- start 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.149+0000 7f9915252700 1 -- start start 2023-10-06T01:39:57.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 0x7f9910149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99101598e0 0x7f99100c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100c9b20 con 0x7f9910159510 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910156c40 con 0x7f99100cbcb0 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f9915252700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100c9560 con 0x7f99101598e0 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 0x7f9910149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34006/0 (socket says 172.21.15.98:34006) 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 0x7f9910149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49698/0 (socket says 172.21.15.98:49698) 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990e59c700 1 -- 172.21.15.98:0/723671912 learned_addr learned my addr 172.21.15.98:0/723671912 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.150+0000 7f990f59e700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99101598e0 0x7f99100c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 -- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99101598e0 msgr2=0x7f99100c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99101598e0 0x7f99100c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 -- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 msgr2=0x7f9910149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 0x7f9910149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 -- 172.21.15.98:0/723671912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904009420 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f990e59c700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f990400ea50 tx=0x7f99040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400ee90 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400c850 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9908009ae0 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400cc10 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.151+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9910156ee0 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.152+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99100c7400 con 0x7f9910159510 2023-10-06T01:39:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.152+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9904013470 con 0x7f9910159510 2023-10-06T01:39:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.153+0000 7f98f6ffd700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.153+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9904084b40 con 0x7f9910159510 2023-10-06T01:39:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.153+0000 7f990ed9d700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.153+0000 7f990ed9d700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 0x7f98fc0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9900004fd0 tx=0x7f9900007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.157+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9904051960 con 0x7f9910159510 2023-10-06T01:39:57.388 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:57.386+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9910064ad0 con 0x7f9910159510 2023-10-06T01:39:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.013+0000 7f98f6ffd700 1 -- 172.21.15.98:0/723671912 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9904054f80 con 0x7f9910159510 2023-10-06T01:39:58.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 msgr2=0x7f98fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 0x7f98fc0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9900004fd0 tx=0x7f9900007eb0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 msgr2=0x7f9910146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f990400ea50 tx=0x7f99040091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 shutdown_connections 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99101598e0 0x7f99100c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99100cbcb0 0x7f9910149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 --2- 172.21.15.98:0/723671912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9910159510 0x7f9910146fb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 >> 172.21.15.98:0/723671912 conn(0x7f9910089e90 msgr2=0x7f991014be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 shutdown_connections 2023-10-06T01:39:58.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.016+0000 7f9915252700 1 -- 172.21.15.98:0/723671912 wait complete. 2023-10-06T01:39:58.030 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:39:58.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.143+0000 7f803846c700 1 Processor -- start 2023-10-06T01:39:58.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.143+0000 7f803846c700 1 -- start start 2023-10-06T01:39:58.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f803846c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f803846c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f80300b07a0 0x7f80300b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f803846c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300b0ca0 con 0x7f80300b49e0 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f803846c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80300b0f80 con 0x7f80300b07a0 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f803846c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b1260 con 0x7f80301595a0 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f8036a09700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f80300b07a0 0x7f80300b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f8035a07700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f80300b49e0 0x7f80300aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36374/0 (socket says 172.21.15.98:36374) 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.144+0000 7f8035a07700 1 -- 172.21.15.98:0/1302766512 learned_addr learned my addr 172.21.15.98:0/1302766512 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036208700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036a09700 1 -- 172.21.15.98:0/1302766512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 msgr2=0x7f80300aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036a09700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036a09700 1 -- 172.21.15.98:0/1302766512 >> v1:172.21.15.98:6789/0 conn(0x7f80300b49e0 legacy=0x7f80300aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8035206700 1 -- 172.21.15.98:0/1302766512 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467218560 0 0) 0x7f80300b0ca0 con 0x7f80300b49e0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036a09700 1 -- 172.21.15.98:0/1302766512 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300b1540 con 0x7f80300b07a0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8036a09700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b07a0 0x7f80300b1c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80280098e0 tx=0x7f80280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9857d87cfd2ef2f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8035206700 1 -- 172.21.15.98:0/1302766512 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028009e80 con 0x7f80300b07a0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8035206700 1 -- 172.21.15.98:0/1302766512 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800b860 con 0x7f80300b07a0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f8035206700 1 -- 172.21.15.98:0/1302766512 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800bbe0 con 0x7f80300b07a0 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 -- 172.21.15.98:0/1302766512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b07a0 msgr2=0x7f80300b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b07a0 0x7f80300b1c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80280098e0 tx=0x7f80280091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 -- 172.21.15.98:0/1302766512 shutdown_connections 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b07a0 0x7f80300b1c00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 --2- 172.21.15.98:0/1302766512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 -- 172.21.15.98:0/1302766512 >> 172.21.15.98:0/1302766512 conn(0x7f8030099830 msgr2=0x7f803006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.145+0000 7f803846c700 1 -- 172.21.15.98:0/1302766512 shutdown_connections 2023-10-06T01:39:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.146+0000 7f803846c700 1 -- 172.21.15.98:0/1302766512 wait complete. 2023-10-06T01:39:58.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.146+0000 7f803846c700 1 Processor -- start 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 -- start start 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 0x7f803016a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803016aad0 con 0x7f80300b49e0 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803016c070 con 0x7f80300b07a0 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f803846c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803016b460 con 0x7f80301595a0 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f8035a07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f8035a07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34022/0 (socket says 172.21.15.98:34022) 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.147+0000 7f8035a07700 1 -- 172.21.15.98:0/3196757901 learned_addr learned my addr 172.21.15.98:0/3196757901 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:58.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8036a09700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8036208700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 0x7f803016a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 -- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 msgr2=0x7f80300aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 -- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 msgr2=0x7f803016a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 0x7f803016a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 -- 172.21.15.98:0/3196757901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009420 con 0x7f80300b49e0 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8036a09700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8036208700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 0x7f803016a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f8035a07700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f802400c810 tx=0x7f8024007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024005c00 con 0x7f80300b49e0 2023-10-06T01:39:58.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024005d60 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803016b740 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024007320 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.148+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803016bd00 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.149+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803016c200 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.149+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f802401b400 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.150+0000 7f802e7fc700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 0x7f801c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.150+0000 7f8036208700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 0x7f801c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.150+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f80240844a0 con 0x7f80300b49e0 2023-10-06T01:39:58.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.150+0000 7f8036208700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 0x7f801c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80300cf490 tx=0x7f80200076c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.154+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8024051340 con 0x7f80300b49e0 2023-10-06T01:39:58.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.379+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8030064ad0 con 0x7f80300b49e0 2023-10-06T01:39:58.381 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.379+0000 7f802e7fc700 1 -- 172.21.15.98:0/3196757901 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f8024054960 con 0x7f80300b49e0 2023-10-06T01:39:58.381 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:58.381 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:39:58.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 msgr2=0x7f801c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 0x7f801c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80300cf490 tx=0x7f80200076c0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 msgr2=0x7f803016ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f802400c810 tx=0x7f8024007db0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 shutdown_connections 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f801c066f30 0x7f801c0693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80301595a0 0x7f80300aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80300b07a0 0x7f803016a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 --2- 172.21.15.98:0/3196757901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80300b49e0 0x7f803016ac30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 >> 172.21.15.98:0/3196757901 conn(0x7f8030099830 msgr2=0x7f80300b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 shutdown_connections 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.382+0000 7f803846c700 1 -- 172.21.15.98:0/3196757901 wait complete. 2023-10-06T01:39:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:39:58.395 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:58.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.507+0000 7fe3ede23700 1 Processor -- start 2023-10-06T01:39:58.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.507+0000 7fe3ede23700 1 -- start start 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3ede23700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b07a0 0x7fe3e80ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3ede23700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80b0ca0 con 0x7fe3e81595a0 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3ede23700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80b0f80 con 0x7fe3e80b49e0 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3ede23700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80ab390 con 0x7fe3e80b07a0 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3e77fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe3e81595a0 0x7fe3e80aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36380/0 (socket says 172.21.15.98:36380) 2023-10-06T01:39:58.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3e77fe700 1 -- 172.21.15.98:0/1041173653 learned_addr learned my addr 172.21.15.98:0/1041173653 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3e7fff700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b07a0 0x7fe3e80ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156281231 0 0) 0x7fe3e80b0ca0 con 0x7fe3e81595a0 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.508+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3d0003570 con 0x7fe3e81595a0 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974000322 0 0) 0x7fe3d0003570 con 0x7fe3e81595a0 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b07a0 msgr2=0x7fe3e80ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b07a0 0x7fe3e80ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 >> v1:172.21.15.184:6789/0 conn(0x7fe3e80b49e0 legacy=0x7fe3e80aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80b1260 con 0x7fe3e81595a0 2023-10-06T01:39:58.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fe3d8001970 con 0x7fe3e81595a0 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 >> v1:172.21.15.98:6789/0 conn(0x7fe3e81595a0 legacy=0x7fe3e80aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3d0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d0004810 0x7fe3d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3d00051c0 0x7fe3d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80ab390 con 0x7fe3d0003a90 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d000c050 con 0x7fe3d0004810 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d000c1e0 con 0x7fe3d00051c0 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3d0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e7fff700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b07a0 0x7fe3e80ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:58.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e6ffd700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d0004810 0x7fe3d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 -- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3d00051c0 msgr2=0x7fe3d000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e7fff700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3d00051c0 0x7fe3d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3d00051c0 0x7fe3d000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 -- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d0004810 msgr2=0x7fe3d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d0004810 0x7fe3d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 -- 172.21.15.98:0/1041173653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d0003570 con 0x7fe3d0003a90 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e6ffd700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d0004810 0x7fe3d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:39:58.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e7fff700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3d00051c0 0x7fe3d000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3d0003ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c670 tx=0x7fe3d8007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac2171acdee3e60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8005b70 con 0x7fe3d0003a90 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8005d10 con 0x7fe3d0003a90 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.509+0000 7fe3e67fc700 1 -- 172.21.15.98:0/1041173653 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8007360 con 0x7fe3d0003a90 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 -- 172.21.15.98:0/1041173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 msgr2=0x7fe3d0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 --2- 172.21.15.98:0/1041173653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3d0003ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c670 tx=0x7fe3d8007ca0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3e7fff700 1 -- 172.21.15.98:0/1041173653 reap_dead start 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 -- 172.21.15.98:0/1041173653 shutdown_connections 2023-10-06T01:39:58.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 -- 172.21.15.98:0/1041173653 >> 172.21.15.98:0/1041173653 conn(0x7fe3e8099830 msgr2=0x7fe3e806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 -- 172.21.15.98:0/1041173653 shutdown_connections 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.510+0000 7fe3ede23700 1 -- 172.21.15.98:0/1041173653 wait complete. 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.511+0000 7fe3ede23700 1 Processor -- start 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.511+0000 7fe3ede23700 1 -- start start 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e816d5b0 0x7fe3e816da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80add50 0x7fe3e80ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e809ca70 con 0x7fe3d0003a90 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e816e000 con 0x7fe3e816d5b0 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3ede23700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e816e2e0 con 0x7fe3e80add50 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3e77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34034/0 (socket says 172.21.15.98:34034) 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3e77fe700 1 -- 172.21.15.98:0/5181438 learned_addr learned my addr 172.21.15.98:0/5181438 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3e7fff700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80add50 0x7fe3e80ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.512+0000 7fe3e6ffd700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e816d5b0 0x7fe3e816da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 -- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80add50 msgr2=0x7fe3e80ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80add50 0x7fe3e80ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 -- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e816d5b0 msgr2=0x7fe3e816da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e816d5b0 0x7fe3e816da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 -- 172.21.15.98:0/5181438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e809c700 con 0x7fe3d0003a90 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3e77fe700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00057c0 tx=0x7fe3d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8007320 con 0x7fe3d0003a90 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8001930 con 0x7fe3d0003a90 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e809c0e0 con 0x7fe3d0003a90 2023-10-06T01:39:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8001cb0 con 0x7fe3d0003a90 2023-10-06T01:39:58.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.513+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e8158540 con 0x7fe3d0003a90 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.514+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e809c560 con 0x7fe3d0003a90 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.514+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3d8014630 con 0x7fe3d0003a90 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.515+0000 7fe3cf7fe700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 0x7fe3d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.515+0000 7fe3e6ffd700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 0x7fe3d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.515+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe3d8086900 con 0x7fe3d0003a90 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.515+0000 7fe3e6ffd700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 0x7fe3d40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc000c00 tx=0x7fe3dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.519+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e809c560 con 0x7fe3d0003a90 2023-10-06T01:39:58.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.704+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3e8158540 con 0x7fe3d0003a90 2023-10-06T01:39:58.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.705+0000 7fe3cf7fe700 1 -- 172.21.15.98:0/5181438 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fe3e8158540 con 0x7fe3d0003a90 2023-10-06T01:39:58.707 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:58.708 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:39:58.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 msgr2=0x7fe3d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 0x7fe3d40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc000c00 tx=0x7fe3dc009040 comp rx=0 tx=0).stop 2023-10-06T01:39:58.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 msgr2=0x7fe3e8157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00057c0 tx=0x7fe3d8015000 comp rx=0 tx=0).stop 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 shutdown_connections 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe3d4066f30 0x7fe3d40693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80add50 0x7fe3e80ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d0003a90 0x7fe3e8157e40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 --2- 172.21.15.98:0/5181438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e816d5b0 0x7fe3e816da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.707+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 >> 172.21.15.98:0/5181438 conn(0x7fe3e8099830 msgr2=0x7fe3e8069750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.708+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 shutdown_connections 2023-10-06T01:39:58.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.708+0000 7fe3ede23700 1 -- 172.21.15.98:0/5181438 wait complete. 2023-10-06T01:39:58.721 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T01:39:58.721 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:39:58.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.834+0000 7f14bad3e700 1 Processor -- start 2023-10-06T01:39:58.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.834+0000 7f14bad3e700 1 -- start start 2023-10-06T01:39:58.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14bad3e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 0x7f14b41645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14bad3e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40b0de0 con 0x7f14b40b29f0 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14bad3e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40cbfe0 con 0x7f14b40b2620 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14bad3e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cb3d0 con 0x7f14b40cd480 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14b3fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14b40b29f0 0x7f14b4159160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36382/0 (socket says 172.21.15.98:36382) 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.835+0000 7f14b3fff700 1 -- 172.21.15.98:0/725194072 learned_addr learned my addr 172.21.15.98:0/725194072 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b92db700 1 --2- 172.21.15.98:0/725194072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 0x7f14b41645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729883294 0 0) 0x7f14a0001110 con 0x7f14b40b29f0 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a4003570 con 0x7f14b40b29f0 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655449849 0 0) 0x7f14b40cbfe0 con 0x7f14b40b2620 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a0001110 con 0x7f14b40b2620 2023-10-06T01:39:58.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b92db700 1 -- 172.21.15.98:0/725194072 >> v1:172.21.15.184:6789/0 conn(0x7f14b40b2620 legacy=0x7f14b40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355876427 0 0) 0x7f14a4003570 con 0x7f14b40b29f0 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b92db700 1 -- 172.21.15.98:0/725194072 >> v1:172.21.15.98:6789/0 conn(0x7f14b40b29f0 legacy=0x7f14b4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b92db700 1 -- 172.21.15.98:0/725194072 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40cb6b0 con 0x7f14b40cd480 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b92db700 1 --2- 172.21.15.98:0/725194072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 0x7f14b41645e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009e90 tx=0x7f14ac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50ad841c1cebdea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e070 con 0x7f14b40cd480 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00b870 con 0x7f14b40cd480 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.836+0000 7f14b37fe700 1 -- 172.21.15.98:0/725194072 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0001600 con 0x7f14b40cd480 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 -- 172.21.15.98:0/725194072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 msgr2=0x7f14b41645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 --2- 172.21.15.98:0/725194072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 0x7f14b41645e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009e90 tx=0x7f14ac009780 comp rx=0 tx=0).stop 2023-10-06T01:39:58.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 -- 172.21.15.98:0/725194072 shutdown_connections 2023-10-06T01:39:58.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 --2- 172.21.15.98:0/725194072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40cd480 0x7f14b41645e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 -- 172.21.15.98:0/725194072 >> 172.21.15.98:0/725194072 conn(0x7f14b406ed20 msgr2=0x7f14b406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:58.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 -- 172.21.15.98:0/725194072 shutdown_connections 2023-10-06T01:39:58.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.837+0000 7f14bad3e700 1 -- 172.21.15.98:0/725194072 wait complete. 2023-10-06T01:39:58.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.838+0000 7f14bad3e700 1 Processor -- start 2023-10-06T01:39:58.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.838+0000 7f14bad3e700 1 -- start start 2023-10-06T01:39:58.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40b2620 0x7f14b40c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b40b29f0 0x7f14b414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414da10 con 0x7f14b40cd480 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414dcf0 con 0x7f14b40b29f0 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14bad3e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414dfd0 con 0x7f14b40b2620 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34050/0 (socket says 172.21.15.98:34050) 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 -- 172.21.15.98:0/1823242853 learned_addr learned my addr 172.21.15.98:0/1823242853 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b3fff700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b40b29f0 0x7f14b414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b8ada700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40b2620 0x7f14b40c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 -- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40b2620 msgr2=0x7f14b40c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40b2620 0x7f14b40c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 -- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b40b29f0 msgr2=0x7f14b414b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b40b29f0 0x7f14b414b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 -- 172.21.15.98:0/1823242853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009a20 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.839+0000 7f14b92db700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009de0 tx=0x7f14ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac012c20 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b414e2b0 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac012d80 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac01e6a0 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b40c6e80 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.840+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4064ad0 con 0x7f14b40cd480 2023-10-06T01:39:58.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.841+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14ac01ead0 con 0x7f14b40cd480 2023-10-06T01:39:58.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.841+0000 7f14b0ff9700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 0x7f14980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:58.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.841+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f14ac0867f0 con 0x7f14b40cd480 2023-10-06T01:39:58.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.841+0000 7f14b8ada700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 0x7f14980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:58.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.842+0000 7f14b8ada700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 0x7f14980693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14a8003ae0 tx=0x7f14a8007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:58.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:58.845+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14ac053610 con 0x7f14b40cd480 2023-10-06T01:39:59.026 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.024+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14b40cb990 con 0x7f1498066f30 2023-10-06T01:39:59.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.035+0000 7f14b0ff9700 1 -- 172.21.15.98:0/1823242853 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367627 (secure 0 0 0) 0x7f14b40cb990 con 0x7f1498066f30 2023-10-06T01:39:59.041 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 msgr2=0x7f14980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 0x7f14980693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14a8003ae0 tx=0x7f14a8007d40 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 msgr2=0x7f14b41697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009de0 tx=0x7f14ac017000 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 shutdown_connections 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1498066f30 0x7f14980693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b40cd480 0x7f14b41697e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b40b2620 0x7f14b40c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 --2- 172.21.15.98:0/1823242853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b40b29f0 0x7f14b414b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.038+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 >> 172.21.15.98:0/1823242853 conn(0x7f14b406ed20 msgr2=0x7f14b40b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.039+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 shutdown_connections 2023-10-06T01:39:59.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.039+0000 7f14bad3e700 1 -- 172.21.15.98:0/1823242853 wait complete. 2023-10-06T01:39:59.045 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:39:59.052 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T01:39:57.787301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":327,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2021678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.451112"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346761+0000","last_change":"2023-10-06T01:39:45.335803+0000","last_active":"2023-10-06T01:39:49.346761+0000","last_peered":"2023-10-06T01:39:49.346761+0000","last_clean":"2023-10-06T01:39:49.346761+0000","last_became_active":"2023-10-06T01:39:45.335420+0000","last_became_peered":"2023-10-06T01:39:45.335420+0000","last_unstale":"2023-10-06T01:39:49.346761+0000","last_undegraded":"2023-10-06T01:39:49.346761+0000","last_fullsized":"2023-10-06T01:39:49.346761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:58.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353326+0000","last_change":"2023-10-06T01:39:45.336392+0000","last_active":"2023-10-06T01:39:49.353326+0000","last_peered":"2023-10-06T01:39:49.353326+0000","last_clean":"2023-10-06T01:39:49.353326+0000","last_became_active":"2023-10-06T01:39:45.336114+0000","last_became_peered":"2023-10-06T01:39:45.336114+0000","last_unstale":"2023-10-06T01:39:49.353326+0000","last_undegraded":"2023-10-06T01:39:49.353326+0000","last_fullsized":"2023-10-06T01:39:49.353326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340003+0000","last_change":"2023-10-06T01:39:45.330889+0000","last_active":"2023-10-06T01:39:49.340003+0000","last_peered":"2023-10-06T01:39:49.340003+0000","last_clean":"2023-10-06T01:39:49.340003+0000","last_became_active":"2023-10-06T01:39:45.330584+0000","last_became_peered":"2023-10-06T01:39:45.330584+0000","last_unstale":"2023-10-06T01:39:49.340003+0000","last_undegraded":"2023-10-06T01:39:49.340003+0000","last_fullsized":"2023-10-06T01:39:49.340003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:55.095137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340381+0000","last_change":"2023-10-06T01:39:45.327678+0000","last_active":"2023-10-06T01:39:49.340381+0000","last_peered":"2023-10-06T01:39:49.340381+0000","last_clean":"2023-10-06T01:39:49.340381+0000","last_became_active":"2023-10-06T01:39:45.327329+0000","last_became_peered":"2023-10-06T01:39:45.327329+0000","last_unstale":"2023-10-06T01:39:49.340381+0000","last_undegraded":"2023-10-06T01:39:49.340381+0000","last_fullsized":"2023-10-06T01:39:49.340381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:13.570152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347538+0000","last_change":"2023-10-06T01:39:45.342805+0000","last_active":"2023-10-06T01:39:49.347538+0000","last_peered":"2023-10-06T01:39:49.347538+0000","last_clean":"2023-10-06T01:39:49.347538+0000","last_became_active":"2023-10-06T01:39:45.342510+0000","last_became_peered":"2023-10-06T01:39:45.342510+0000","last_unstale":"2023-10-06T01:39:49.347538+0000","last_undegraded":"2023-10-06T01:39:49.347538+0000","last_fullsized":"2023-10-06T01:39:49.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340791+0000","last_change":"2023-10-06T01:39:45.328521+0000","last_active":"2023-10-06T01:39:49.340791+0000","last_peered":"2023-10-06T01:39:49.340791+0000","last_clean":"2023-10-06T01:39:49.340791+0000","last_became_active":"2023-10-06T01:39:45.327249+0000","last_became_peered":"2023-10-06T01:39:45.327249+0000","last_unstale":"2023-10-06T01:39:49.340791+0000","last_undegraded":"2023-10-06T01:39:49.340791+0000","last_fullsized":"2023-10-06T01:39:49.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.886175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350835+0000","last_change":"2023-10-06T01:39:45.336408+0000","last_active":"2023-10-06T01:39:49.350835+0000","last_peered":"2023-10-06T01:39:49.350835+0000","last_clean":"2023-10-06T01:39:49.350835+0000","last_became_active":"2023-10-06T01:39:45.336066+0000","last_became_peered":"2023-10-06T01:39:45.336066+0000","last_unstale":"2023-10-06T01:39:49.350835+0000","last_undegraded":"2023-10-06T01:39:49.350835+0000","last_fullsized":"2023-10-06T01:39:49.350835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350056+0000","last_change":"2023-10-06T01:39:45.331003+0000","last_active":"2023-10-06T01:39:49.350056+0000","last_peered":"2023-10-06T01:39:49.350056+0000","last_clean":"2023-10-06T01:39:49.350056+0000","last_became_active":"2023-10-06T01:39:45.330563+0000","last_became_peered":"2023-10-06T01:39:45.330563+0000","last_unstale":"2023-10-06T01:39:49.350056+0000","last_undegraded":"2023-10-06T01:39:49.350056+0000","last_fullsized":"2023-10-06T01:39:49.350056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351292+0000","last_change":"2023-10-06T01:39:45.344173+0000","last_active":"2023-10-06T01:39:49.351292+0000","last_peered":"2023-10-06T01:39:49.351292+0000","last_clean":"2023-10-06T01:39:49.351292+0000","last_became_active":"2023-10-06T01:39:45.343891+0000","last_became_peered":"2023-10-06T01:39:45.343891+0000","last_unstale":"2023-10-06T01:39:49.351292+0000","last_undegraded":"2023-10-06T01:39:49.351292+0000","last_fullsized":"2023-10-06T01:39:49.351292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353439+0000","last_change":"2023-10-06T01:39:45.332773+0000","last_active":"2023-10-06T01:39:49.353439+0000","last_peered":"2023-10-06T01:39:49.353439+0000","last_clean":"2023-10-06T01:39:49.353439+0000","last_became_active":"2023-10-06T01:39:45.332504+0000","last_became_peered":"2023-10-06T01:39:45.332504+0000","last_unstale":"2023-10-06T01:39:49.353439+0000","last_undegraded":"2023-10-06T01:39:49.353439+0000","last_fullsized":"2023-10-06T01:39:49.353439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340177+0000","last_change":"2023-10-06T01:39:45.335528+0000","last_active":"2023-10-06T01:39:49.340177+0000","last_peered":"2023-10-06T01:39:49.340177+0000","last_clean":"2023-10-06T01:39:49.340177+0000","last_became_active":"2023-10-06T01:39:45.335296+0000","last_became_peered":"2023-10-06T01:39:45.335296+0000","last_unstale":"2023-10-06T01:39:49.340177+0000","last_undegraded":"2023-10-06T01:39:49.340177+0000","last_fullsized":"2023-10-06T01:39:49.340177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:09.004117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346177+0000","last_change":"2023-10-06T01:39:45.326855+0000","last_active":"2023-10-06T01:39:49.346177+0000","last_peered":"2023-10-06T01:39:49.346177+0000","last_clean":"2023-10-06T01:39:49.346177+0000","last_became_active":"2023-10-06T01:39:45.326260+0000","last_became_peered":"2023-10-06T01:39:45.326260+0000","last_unstale":"2023-10-06T01:39:49.346177+0000","last_undegraded":"2023-10-06T01:39:49.346177+0000","last_fullsized":"2023-10-06T01:39:49.346177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:26.407500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345996+0000","last_change":"2023-10-06T01:39:45.346287+0000","last_active":"2023-10-06T01:39:49.345996+0000","last_peered":"2023-10-06T01:39:49.345996+0000","last_clean":"2023-10-06T01:39:49.345996+0000","last_became_active":"2023-10-06T01:39:45.346017+0000","last_became_peered":"2023-10-06T01:39:45.346017+0000","last_unstale":"2023-10-06T01:39:49.345996+0000","last_undegraded":"2023-10-06T01:39:49.345996+0000","last_fullsized":"2023-10-06T01:39:49.345996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:37.090919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350016+0000","last_change":"2023-10-06T01:39:45.337191+0000","last_active":"2023-10-06T01:39:49.350016+0000","last_peered":"2023-10-06T01:39:49.350016+0000","last_clean":"2023-10-06T01:39:49.350016+0000","last_became_active":"2023-10-06T01:39:45.336905+0000","last_became_peered":"2023-10-06T01:39:45.336905+0000","last_unstale":"2023-10-06T01:39:49.350016+0000","last_undegraded":"2023-10-06T01:39:49.350016+0000","last_fullsized":"2023-10-06T01:39:49.350016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350613+0000","last_change":"2023-10-06T01:39:45.336653+0000","last_active":"2023-10-06T01:39:49.350613+0000","last_peered":"2023-10-06T01:39:49.350613+0000","last_clean":"2023-10-06T01:39:49.350613+0000","last_became_active":"2023-10-06T01:39:45.336383+0000","last_became_peered":"2023-10-06T01:39:45.336383+0000","last_unstale":"2023-10-06T01:39:49.350613+0000","last_undegraded":"2023-10-06T01:39:49.350613+0000","last_fullsized":"2023-10-06T01:39:49.350613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342395+0000","last_change":"2023-10-06T01:39:45.332271+0000","last_active":"2023-10-06T01:39:49.342395+0000","last_peered":"2023-10-06T01:39:49.342395+0000","last_clean":"2023-10-06T01:39:49.342395+0000","last_became_active":"2023-10-06T01:39:45.331844+0000","last_became_peered":"2023-10-06T01:39:45.331844+0000","last_unstale":"2023-10-06T01:39:49.342395+0000","last_undegraded":"2023-10-06T01:39:49.342395+0000","last_fullsized":"2023-10-06T01:39:49.342395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:01.791676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351382+0000","last_change":"2023-10-06T01:39:45.336079+0000","last_active":"2023-10-06T01:39:49.351382+0000","last_peered":"2023-10-06T01:39:49.351382+0000","last_clean":"2023-10-06T01:39:49.351382+0000","last_became_active":"2023-10-06T01:39:45.335828+0000","last_became_peered":"2023-10-06T01:39:45.335828+0000","last_unstale":"2023-10-06T01:39:49.351382+0000","last_undegraded":"2023-10-06T01:39:49.351382+0000","last_fullsized":"2023-10-06T01:39:49.351382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342168+0000","last_change":"2023-10-06T01:39:45.339626+0000","last_active":"2023-10-06T01:39:49.342168+0000","last_peered":"2023-10-06T01:39:49.342168+0000","last_clean":"2023-10-06T01:39:49.342168+0000","last_became_active":"2023-10-06T01:39:45.339370+0000","last_became_peered":"2023-10-06T01:39:45.339370+0000","last_unstale":"2023-10-06T01:39:49.342168+0000","last_undegraded":"2023-10-06T01:39:49.342168+0000","last_fullsized":"2023-10-06T01:39:49.342168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:12.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346086+0000","last_change":"2023-10-06T01:39:45.325821+0000","last_active":"2023-10-06T01:39:49.346086+0000","last_peered":"2023-10-06T01:39:49.346086+0000","last_clean":"2023-10-06T01:39:49.346086+0000","last_became_active":"2023-10-06T01:39:45.324033+0000","last_became_peered":"2023-10-06T01:39:45.324033+0000","last_unstale":"2023-10-06T01:39:49.346086+0000","last_undegraded":"2023-10-06T01:39:49.346086+0000","last_fullsized":"2023-10-06T01:39:49.346086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:13.558462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:45.330675+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:45.330406+0000","last_became_peered":"2023-10-06T01:39:45.330406+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350748+0000","last_change":"2023-10-06T01:39:45.342284+0000","last_active":"2023-10-06T01:39:49.350748+0000","last_peered":"2023-10-06T01:39:49.350748+0000","last_clean":"2023-10-06T01:39:49.350748+0000","last_became_active":"2023-10-06T01:39:45.342034+0000","last_became_peered":"2023-10-06T01:39:45.342034+0000","last_unstale":"2023-10-06T01:39:49.350748+0000","last_undegraded":"2023-10-06T01:39:49.350748+0000","last_fullsized":"2023-10-06T01:39:49.350748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347974+0000","last_change":"2023-10-06T01:39:45.334697+0000","last_active":"2023-10-06T01:39:49.347974+0000","last_peered":"2023-10-06T01:39:49.347974+0000","last_clean":"2023-10-06T01:39:49.347974+0000","last_became_active":"2023-10-06T01:39:45.334398+0000","last_became_peered":"2023-10-06T01:39:45.334398+0000","last_unstale":"2023-10-06T01:39:49.347974+0000","last_undegraded":"2023-10-06T01:39:49.347974+0000","last_fullsized":"2023-10-06T01:39:49.347974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338438+0000","last_change":"2023-10-06T01:39:45.336938+0000","last_active":"2023-10-06T01:39:49.338438+0000","last_peered":"2023-10-06T01:39:49.338438+0000","last_clean":"2023-10-06T01:39:49.338438+0000","last_became_active":"2023-10-06T01:39:45.336679+0000","last_became_peered":"2023-10-06T01:39:45.336679+0000","last_unstale":"2023-10-06T01:39:49.338438+0000","last_undegraded":"2023-10-06T01:39:49.338438+0000","last_fullsized":"2023-10-06T01:39:49.338438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350674+0000","last_change":"2023-10-06T01:39:45.338035+0000","last_active":"2023-10-06T01:39:49.350674+0000","last_peered":"2023-10-06T01:39:49.350674+0000","last_clean":"2023-10-06T01:39:49.350674+0000","last_became_active":"2023-10-06T01:39:45.337835+0000","last_became_peered":"2023-10-06T01:39:45.337835+0000","last_unstale":"2023-10-06T01:39:49.350674+0000","last_undegraded":"2023-10-06T01:39:49.350674+0000","last_fullsized":"2023-10-06T01:39:49.350674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344202+0000","last_change":"2023-10-06T01:39:45.328764+0000","last_active":"2023-10-06T01:39:49.344202+0000","last_peered":"2023-10-06T01:39:49.344202+0000","last_clean":"2023-10-06T01:39:49.344202+0000","last_became_active":"2023-10-06T01:39:45.328221+0000","last_became_peered":"2023-10-06T01:39:45.328221+0000","last_unstale":"2023-10-06T01:39:49.344202+0000","last_undegraded":"2023-10-06T01:39:49.344202+0000","last_fullsized":"2023-10-06T01:39:49.344202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344090+0000","last_change":"2023-10-06T01:39:45.337344+0000","last_active":"2023-10-06T01:39:49.344090+0000","last_peered":"2023-10-06T01:39:49.344090+0000","last_clean":"2023-10-06T01:39:49.344090+0000","last_became_active":"2023-10-06T01:39:45.336952+0000","last_became_peered":"2023-10-06T01:39:45.336952+0000","last_unstale":"2023-10-06T01:39:49.344090+0000","last_undegraded":"2023-10-06T01:39:49.344090+0000","last_fullsized":"2023-10-06T01:39:49.344090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350190+0000","last_change":"2023-10-06T01:39:45.334194+0000","last_active":"2023-10-06T01:39:49.350190+0000","last_peered":"2023-10-06T01:39:49.350190+0000","last_clean":"2023-10-06T01:39:49.350190+0000","last_became_active":"2023-10-06T01:39:45.333745+0000","last_became_peered":"2023-10-06T01:39:45.333745+0000","last_unstale":"2023-10-06T01:39:49.350190+0000","last_undegraded":"2023-10-06T01:39:49.350190+0000","last_fullsized":"2023-10-06T01:39:49.350190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360851+0000","last_change":"2023-10-06T01:39:49.359145+0000","last_active":"2023-10-06T01:39:49.360851+0000","last_peered":"2023-10-06T01:39:49.360851+0000","last_clean":"2023-10-06T01:39:49.360851+0000","last_became_active":"2023-10-06T01:39:49.358785+0000","last_became_peered":"2023-10-06T01:39:49.358785+0000","last_unstale":"2023-10-06T01:39:49.360851+0000","last_undegraded":"2023-10-06T01:39:49.360851+0000","last_fullsized":"2023-10-06T01:39:49.360851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:22.255025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340582+0000","last_change":"2023-10-06T01:39:44.325257+0000","last_active":"2023-10-06T01:39:49.340582+0000","last_peered":"2023-10-06T01:39:49.340582+0000","last_clean":"2023-10-06T01:39:49.340582+0000","last_became_active":"2023-10-06T01:39:44.324956+0000","last_became_peered":"2023-10-06T01:39:44.324956+0000","last_unstale":"2023-10-06T01:39:49.340582+0000","last_undegraded":"2023-10-06T01:39:49.340582+0000","last_fullsized":"2023-10-06T01:39:49.340582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338359+0000","last_change":"2023-10-06T01:39:45.331219+0000","last_active":"2023-10-06T01:39:49.338359+0000","last_peered":"2023-10-06T01:39:49.338359+0000","last_clean":"2023-10-06T01:39:49.338359+0000","last_became_active":"2023-10-06T01:39:45.330964+0000","last_became_peered":"2023-10-06T01:39:45.330964+0000","last_unstale":"2023-10-06T01:39:49.338359+0000","last_undegraded":"2023-10-06T01:39:49.338359+0000","last_fullsized":"2023-10-06T01:39:49.338359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359624+0000","last_change":"2023-10-06T01:39:49.356248+0000","last_active":"2023-10-06T01:39:49.359624+0000","last_peered":"2023-10-06T01:39:49.359624+0000","last_clean":"2023-10-06T01:39:49.359624+0000","last_became_active":"2023-10-06T01:39:49.355851+0000","last_became_peered":"2023-10-06T01:39:49.355851+0000","last_unstale":"2023-10-06T01:39:49.359624+0000","last_undegraded":"2023-10-06T01:39:49.359624+0000","last_fullsized":"2023-10-06T01:39:49.359624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341956+0000","last_change":"2023-10-06T01:39:44.327847+0000","last_active":"2023-10-06T01:39:49.341956+0000","last_peered":"2023-10-06T01:39:49.341956+0000","last_clean":"2023-10-06T01:39:49.341956+0000","last_became_active":"2023-10-06T01:39:44.327533+0000","last_became_peered":"2023-10-06T01:39:44.327533+0000","last_unstale":"2023-10-06T01:39:49.341956+0000","last_undegraded":"2023-10-06T01:39:49.341956+0000","last_fullsized":"2023-10-06T01:39:49.341956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351266+0000","last_change":"2023-10-06T01:39:45.326239+0000","last_active":"2023-10-06T01:39:49.351266+0000","last_peered":"2023-10-06T01:39:49.351266+0000","last_clean":"2023-10-06T01:39:49.351266+0000","last_became_active":"2023-10-06T01:39:45.325981+0000","last_became_peered":"2023-10-06T01:39:45.325981+0000","last_unstale":"2023-10-06T01:39:49.351266+0000","last_undegraded":"2023-10-06T01:39:49.351266+0000","last_fullsized":"2023-10-06T01:39:49.351266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361578+0000","last_change":"2023-10-06T01:39:49.358502+0000","last_active":"2023-10-06T01:39:49.361578+0000","last_peered":"2023-10-06T01:39:49.361578+0000","last_clean":"2023-10-06T01:39:49.361578+0000","last_became_active":"2023-10-06T01:39:49.358114+0000","last_became_peered":"2023-10-06T01:39:49.358114+0000","last_unstale":"2023-10-06T01:39:49.361578+0000","last_undegraded":"2023-10-06T01:39:49.361578+0000","last_fullsized":"2023-10-06T01:39:49.361578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338202+0000","last_change":"2023-10-06T01:39:44.325114+0000","last_active":"2023-10-06T01:39:49.338202+0000","last_peered":"2023-10-06T01:39:49.338202+0000","last_clean":"2023-10-06T01:39:49.338202+0000","last_became_active":"2023-10-06T01:39:44.324802+0000","last_became_peered":"2023-10-06T01:39:44.324802+0000","last_unstale":"2023-10-06T01:39:49.338202+0000","last_undegraded":"2023-10-06T01:39:49.338202+0000","last_fullsized":"2023-10-06T01:39:49.338202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:25.402016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339917+0000","last_change":"2023-10-06T01:39:45.337997+0000","last_active":"2023-10-06T01:39:49.339917+0000","last_peered":"2023-10-06T01:39:49.339917+0000","last_clean":"2023-10-06T01:39:49.339917+0000","last_became_active":"2023-10-06T01:39:45.337770+0000","last_became_peered":"2023-10-06T01:39:45.337770+0000","last_unstale":"2023-10-06T01:39:49.339917+0000","last_undegraded":"2023-10-06T01:39:49.339917+0000","last_fullsized":"2023-10-06T01:39:49.339917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359080+0000","last_change":"2023-10-06T01:39:49.355683+0000","last_active":"2023-10-06T01:39:49.359080+0000","last_peered":"2023-10-06T01:39:49.359080+0000","last_clean":"2023-10-06T01:39:49.359080+0000","last_became_active":"2023-10-06T01:39:49.355122+0000","last_became_peered":"2023-10-06T01:39:49.355122+0000","last_unstale":"2023-10-06T01:39:49.359080+0000","last_undegraded":"2023-10-06T01:39:49.359080+0000","last_fullsized":"2023-10-06T01:39:49.359080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351467+0000","last_change":"2023-10-06T01:39:44.326264+0000","last_active":"2023-10-06T01:39:49.351467+0000","last_peered":"2023-10-06T01:39:49.351467+0000","last_clean":"2023-10-06T01:39:49.351467+0000","last_became_active":"2023-10-06T01:39:44.325998+0000","last_became_peered":"2023-10-06T01:39:44.325998+0000","last_unstale":"2023-10-06T01:39:49.351467+0000","last_undegraded":"2023-10-06T01:39:49.351467+0000","last_fullsized":"2023-10-06T01:39:49.351467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338117+0000","last_change":"2023-10-06T01:39:45.332320+0000","last_active":"2023-10-06T01:39:49.338117+0000","last_peered":"2023-10-06T01:39:49.338117+0000","last_clean":"2023-10-06T01:39:49.338117+0000","last_became_active":"2023-10-06T01:39:45.332049+0000","last_became_peered":"2023-10-06T01:39:45.332049+0000","last_unstale":"2023-10-06T01:39:49.338117+0000","last_undegraded":"2023-10-06T01:39:49.338117+0000","last_fullsized":"2023-10-06T01:39:49.338117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363738+0000","last_change":"2023-10-06T01:39:49.360295+0000","last_active":"2023-10-06T01:39:49.363738+0000","last_peered":"2023-10-06T01:39:49.363738+0000","last_clean":"2023-10-06T01:39:49.363738+0000","last_became_active":"2023-10-06T01:39:49.359873+0000","last_became_peered":"2023-10-06T01:39:49.359873+0000","last_unstale":"2023-10-06T01:39:49.363738+0000","last_undegraded":"2023-10-06T01:39:49.363738+0000","last_fullsized":"2023-10-06T01:39:49.363738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356499+0000","last_change":"2023-10-06T01:39:44.328151+0000","last_active":"2023-10-06T01:39:49.356499+0000","last_peered":"2023-10-06T01:39:49.356499+0000","last_clean":"2023-10-06T01:39:49.356499+0000","last_became_active":"2023-10-06T01:39:44.327876+0000","last_became_peered":"2023-10-06T01:39:44.327876+0000","last_unstale":"2023-10-06T01:39:49.356499+0000","last_undegraded":"2023-10-06T01:39:49.356499+0000","last_fullsized":"2023-10-06T01:39:49.356499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337479+0000","last_change":"2023-10-06T01:39:45.330616+0000","last_active":"2023-10-06T01:39:49.337479+0000","last_peered":"2023-10-06T01:39:49.337479+0000","last_clean":"2023-10-06T01:39:49.337479+0000","last_became_active":"2023-10-06T01:39:45.330340+0000","last_became_peered":"2023-10-06T01:39:45.330340+0000","last_unstale":"2023-10-06T01:39:49.337479+0000","last_undegraded":"2023-10-06T01:39:49.337479+0000","last_fullsized":"2023-10-06T01:39:49.337479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368948+0000","last_change":"2023-10-06T01:39:49.365881+0000","last_active":"2023-10-06T01:39:49.368948+0000","last_peered":"2023-10-06T01:39:49.368948+0000","last_clean":"2023-10-06T01:39:49.368948+0000","last_became_active":"2023-10-06T01:39:49.365499+0000","last_became_peered":"2023-10-06T01:39:49.365499+0000","last_unstale":"2023-10-06T01:39:49.368948+0000","last_undegraded":"2023-10-06T01:39:49.368948+0000","last_fullsized":"2023-10-06T01:39:49.368948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:06.882659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351163+0000","last_change":"2023-10-06T01:39:44.326937+0000","last_active":"2023-10-06T01:39:49.351163+0000","last_peered":"2023-10-06T01:39:49.351163+0000","last_clean":"2023-10-06T01:39:49.351163+0000","last_became_active":"2023-10-06T01:39:44.326701+0000","last_became_peered":"2023-10-06T01:39:44.326701+0000","last_unstale":"2023-10-06T01:39:49.351163+0000","last_undegraded":"2023-10-06T01:39:49.351163+0000","last_fullsized":"2023-10-06T01:39:49.351163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:23.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342612+0000","last_change":"2023-10-06T01:39:45.331176+0000","last_active":"2023-10-06T01:39:49.342612+0000","last_peered":"2023-10-06T01:39:49.342612+0000","last_clean":"2023-10-06T01:39:49.342612+0000","last_became_active":"2023-10-06T01:39:45.330927+0000","last_became_peered":"2023-10-06T01:39:45.330927+0000","last_unstale":"2023-10-06T01:39:49.342612+0000","last_undegraded":"2023-10-06T01:39:49.342612+0000","last_fullsized":"2023-10-06T01:39:49.342612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357166+0000","last_change":"2023-10-06T01:39:49.355332+0000","last_active":"2023-10-06T01:39:49.357166+0000","last_peered":"2023-10-06T01:39:49.357166+0000","last_clean":"2023-10-06T01:39:49.357166+0000","last_became_active":"2023-10-06T01:39:49.354961+0000","last_became_peered":"2023-10-06T01:39:49.354961+0000","last_unstale":"2023-10-06T01:39:49.357166+0000","last_undegraded":"2023-10-06T01:39:49.357166+0000","last_fullsized":"2023-10-06T01:39:49.357166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T01:39:55.162081+0000","last_change":"2023-10-06T01:39:44.325901+0000","last_active":"2023-10-06T01:39:55.162081+0000","last_peered":"2023-10-06T01:39:55.162081+0000","last_clean":"2023-10-06T01:39:55.162081+0000","last_became_active":"2023-10-06T01:39:44.325631+0000","last_became_peered":"2023-10-06T01:39:44.325631+0000","last_unstale":"2023-10-06T01:39:55.162081+0000","last_undegraded":"2023-10-06T01:39:55.162081+0000","last_fullsized":"2023-10-06T01:39:55.162081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340321+0000","last_change":"2023-10-06T01:39:45.328644+0000","last_active":"2023-10-06T01:39:49.340321+0000","last_peered":"2023-10-06T01:39:49.340321+0000","last_clean":"2023-10-06T01:39:49.340321+0000","last_became_active":"2023-10-06T01:39:45.328367+0000","last_became_peered":"2023-10-06T01:39:45.328367+0000","last_unstale":"2023-10-06T01:39:49.340321+0000","last_undegraded":"2023-10-06T01:39:49.340321+0000","last_fullsized":"2023-10-06T01:39:49.340321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367347+0000","last_change":"2023-10-06T01:39:49.364574+0000","last_active":"2023-10-06T01:39:49.367347+0000","last_peered":"2023-10-06T01:39:49.367347+0000","last_clean":"2023-10-06T01:39:49.367347+0000","last_became_active":"2023-10-06T01:39:49.360749+0000","last_became_peered":"2023-10-06T01:39:49.360749+0000","last_unstale":"2023-10-06T01:39:49.367347+0000","last_undegraded":"2023-10-06T01:39:49.367347+0000","last_fullsized":"2023-10-06T01:39:49.367347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350570+0000","last_change":"2023-10-06T01:39:44.332362+0000","last_active":"2023-10-06T01:39:49.350570+0000","last_peered":"2023-10-06T01:39:49.350570+0000","last_clean":"2023-10-06T01:39:49.350570+0000","last_became_active":"2023-10-06T01:39:44.332082+0000","last_became_peered":"2023-10-06T01:39:44.332082+0000","last_unstale":"2023-10-06T01:39:49.350570+0000","last_undegraded":"2023-10-06T01:39:49.350570+0000","last_fullsized":"2023-10-06T01:39:49.350570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338444+0000","last_change":"2023-10-06T01:39:45.333142+0000","last_active":"2023-10-06T01:39:49.338444+0000","last_peered":"2023-10-06T01:39:49.338444+0000","last_clean":"2023-10-06T01:39:49.338444+0000","last_became_active":"2023-10-06T01:39:45.332880+0000","last_became_peered":"2023-10-06T01:39:45.332880+0000","last_unstale":"2023-10-06T01:39:49.338444+0000","last_undegraded":"2023-10-06T01:39:49.338444+0000","last_fullsized":"2023-10-06T01:39:49.338444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359341+0000","last_change":"2023-10-06T01:39:49.357412+0000","last_active":"2023-10-06T01:39:49.359341+0000","last_peered":"2023-10-06T01:39:49.359341+0000","last_clean":"2023-10-06T01:39:49.359341+0000","last_became_active":"2023-10-06T01:39:49.357031+0000","last_became_peered":"2023-10-06T01:39:49.357031+0000","last_unstale":"2023-10-06T01:39:49.359341+0000","last_undegraded":"2023-10-06T01:39:49.359341+0000","last_fullsized":"2023-10-06T01:39:49.359341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.348119+0000","last_change":"2023-10-06T01:39:44.325262+0000","last_active":"2023-10-06T01:39:49.348119+0000","last_peered":"2023-10-06T01:39:49.348119+0000","last_clean":"2023-10-06T01:39:49.348119+0000","last_became_active":"2023-10-06T01:39:44.325017+0000","last_became_peered":"2023-10-06T01:39:44.325017+0000","last_unstale":"2023-10-06T01:39:49.348119+0000","last_undegraded":"2023-10-06T01:39:49.348119+0000","last_fullsized":"2023-10-06T01:39:49.348119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350108+0000","last_change":"2023-10-06T01:39:45.334022+0000","last_active":"2023-10-06T01:39:49.350108+0000","last_peered":"2023-10-06T01:39:49.350108+0000","last_clean":"2023-10-06T01:39:49.350108+0000","last_became_active":"2023-10-06T01:39:45.333462+0000","last_became_peered":"2023-10-06T01:39:45.333462+0000","last_unstale":"2023-10-06T01:39:49.350108+0000","last_undegraded":"2023-10-06T01:39:49.350108+0000","last_fullsized":"2023-10-06T01:39:49.350108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.358648+0000","last_change":"2023-10-06T01:39:49.357484+0000","last_active":"2023-10-06T01:39:49.358648+0000","last_peered":"2023-10-06T01:39:49.358648+0000","last_clean":"2023-10-06T01:39:49.358648+0000","last_became_active":"2023-10-06T01:39:49.356839+0000","last_became_peered":"2023-10-06T01:39:49.356839+0000","last_unstale":"2023-10-06T01:39:49.358648+0000","last_undegraded":"2023-10-06T01:39:49.358648+0000","last_fullsized":"2023-10-06T01:39:49.358648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:14.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350742+0000","last_change":"2023-10-06T01:39:44.333226+0000","last_active":"2023-10-06T01:39:49.350742+0000","last_peered":"2023-10-06T01:39:49.350742+0000","last_clean":"2023-10-06T01:39:49.350742+0000","last_became_active":"2023-10-06T01:39:44.332920+0000","last_became_peered":"2023-10-06T01:39:44.332920+0000","last_unstale":"2023-10-06T01:39:49.350742+0000","last_undegraded":"2023-10-06T01:39:49.350742+0000","last_fullsized":"2023-10-06T01:39:49.350742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.344914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351529+0000","last_change":"2023-10-06T01:39:45.333240+0000","last_active":"2023-10-06T01:39:49.351529+0000","last_peered":"2023-10-06T01:39:49.351529+0000","last_clean":"2023-10-06T01:39:49.351529+0000","last_became_active":"2023-10-06T01:39:45.332985+0000","last_became_peered":"2023-10-06T01:39:45.332985+0000","last_unstale":"2023-10-06T01:39:49.351529+0000","last_undegraded":"2023-10-06T01:39:49.351529+0000","last_fullsized":"2023-10-06T01:39:49.351529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.364965+0000","last_change":"2023-10-06T01:39:49.360274+0000","last_active":"2023-10-06T01:39:49.364965+0000","last_peered":"2023-10-06T01:39:49.364965+0000","last_clean":"2023-10-06T01:39:49.364965+0000","last_became_active":"2023-10-06T01:39:49.359287+0000","last_became_peered":"2023-10-06T01:39:49.359287+0000","last_unstale":"2023-10-06T01:39:49.364965+0000","last_undegraded":"2023-10-06T01:39:49.364965+0000","last_fullsized":"2023-10-06T01:39:49.364965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:39.297464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343143+0000","last_change":"2023-10-06T01:39:44.322253+0000","last_active":"2023-10-06T01:39:49.343143+0000","last_peered":"2023-10-06T01:39:49.343143+0000","last_clean":"2023-10-06T01:39:49.343143+0000","last_became_active":"2023-10-06T01:39:44.321953+0000","last_became_peered":"2023-10-06T01:39:44.321953+0000","last_unstale":"2023-10-06T01:39:49.343143+0000","last_undegraded":"2023-10-06T01:39:49.343143+0000","last_fullsized":"2023-10-06T01:39:49.343143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338146+0000","last_change":"2023-10-06T01:39:45.343656+0000","last_active":"2023-10-06T01:39:49.338146+0000","last_peered":"2023-10-06T01:39:49.338146+0000","last_clean":"2023-10-06T01:39:49.338146+0000","last_became_active":"2023-10-06T01:39:45.342592+0000","last_became_peered":"2023-10-06T01:39:45.342592+0000","last_unstale":"2023-10-06T01:39:49.338146+0000","last_undegraded":"2023-10-06T01:39:49.338146+0000","last_fullsized":"2023-10-06T01:39:49.338146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365064+0000","last_change":"2023-10-06T01:39:49.363708+0000","last_active":"2023-10-06T01:39:49.365064+0000","last_peered":"2023-10-06T01:39:49.365064+0000","last_clean":"2023-10-06T01:39:49.365064+0000","last_became_active":"2023-10-06T01:39:49.363055+0000","last_became_peered":"2023-10-06T01:39:49.363055+0000","last_unstale":"2023-10-06T01:39:49.365064+0000","last_undegraded":"2023-10-06T01:39:49.365064+0000","last_fullsized":"2023-10-06T01:39:49.365064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352442+0000","last_change":"2023-10-06T01:39:44.333358+0000","last_active":"2023-10-06T01:39:49.352442+0000","last_peered":"2023-10-06T01:39:49.352442+0000","last_clean":"2023-10-06T01:39:49.352442+0000","last_became_active":"2023-10-06T01:39:44.332979+0000","last_became_peered":"2023-10-06T01:39:44.332979+0000","last_unstale":"2023-10-06T01:39:49.352442+0000","last_undegraded":"2023-10-06T01:39:49.352442+0000","last_fullsized":"2023-10-06T01:39:49.352442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349928+0000","last_change":"2023-10-06T01:39:45.330806+0000","last_active":"2023-10-06T01:39:49.349928+0000","last_peered":"2023-10-06T01:39:49.349928+0000","last_clean":"2023-10-06T01:39:49.349928+0000","last_became_active":"2023-10-06T01:39:45.330280+0000","last_became_peered":"2023-10-06T01:39:45.330280+0000","last_unstale":"2023-10-06T01:39:49.349928+0000","last_undegraded":"2023-10-06T01:39:49.349928+0000","last_fullsized":"2023-10-06T01:39:49.349928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.355843+0000","last_change":"2023-10-06T01:39:49.352785+0000","last_active":"2023-10-06T01:39:49.355843+0000","last_peered":"2023-10-06T01:39:49.355843+0000","last_clean":"2023-10-06T01:39:49.355843+0000","last_became_active":"2023-10-06T01:39:49.352396+0000","last_became_peered":"2023-10-06T01:39:49.352396+0000","last_unstale":"2023-10-06T01:39:49.355843+0000","last_undegraded":"2023-10-06T01:39:49.355843+0000","last_fullsized":"2023-10-06T01:39:49.355843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:45.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352263+0000","last_change":"2023-10-06T01:39:44.332277+0000","last_active":"2023-10-06T01:39:49.352263+0000","last_peered":"2023-10-06T01:39:49.352263+0000","last_clean":"2023-10-06T01:39:49.352263+0000","last_became_active":"2023-10-06T01:39:44.331987+0000","last_became_peered":"2023-10-06T01:39:44.331987+0000","last_unstale":"2023-10-06T01:39:49.352263+0000","last_undegraded":"2023-10-06T01:39:49.352263+0000","last_fullsized":"2023-10-06T01:39:49.352263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:36.012185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342030+0000","last_change":"2023-10-06T01:39:45.326893+0000","last_active":"2023-10-06T01:39:49.342030+0000","last_peered":"2023-10-06T01:39:49.342030+0000","last_clean":"2023-10-06T01:39:49.342030+0000","last_became_active":"2023-10-06T01:39:45.326610+0000","last_became_peered":"2023-10-06T01:39:45.326610+0000","last_unstale":"2023-10-06T01:39:49.342030+0000","last_undegraded":"2023-10-06T01:39:49.342030+0000","last_fullsized":"2023-10-06T01:39:49.342030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360816+0000","last_change":"2023-10-06T01:39:49.359772+0000","last_active":"2023-10-06T01:39:49.360816+0000","last_peered":"2023-10-06T01:39:49.360816+0000","last_clean":"2023-10-06T01:39:49.360816+0000","last_became_active":"2023-10-06T01:39:49.359414+0000","last_became_peered":"2023-10-06T01:39:49.359414+0000","last_unstale":"2023-10-06T01:39:49.360816+0000","last_undegraded":"2023-10-06T01:39:49.360816+0000","last_fullsized":"2023-10-06T01:39:49.360816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351061+0000","last_change":"2023-10-06T01:39:44.330033+0000","last_active":"2023-10-06T01:39:49.351061+0000","last_peered":"2023-10-06T01:39:49.351061+0000","last_clean":"2023-10-06T01:39:49.351061+0000","last_became_active":"2023-10-06T01:39:44.329732+0000","last_became_peered":"2023-10-06T01:39:44.329732+0000","last_unstale":"2023-10-06T01:39:49.351061+0000","last_undegraded":"2023-10-06T01:39:49.351061+0000","last_fullsized":"2023-10-06T01:39:49.351061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339868+0000","last_change":"2023-10-06T01:39:45.334185+0000","last_active":"2023-10-06T01:39:49.339868+0000","last_peered":"2023-10-06T01:39:49.339868+0000","last_clean":"2023-10-06T01:39:49.339868+0000","last_became_active":"2023-10-06T01:39:45.333053+0000","last_became_peered":"2023-10-06T01:39:45.333053+0000","last_unstale":"2023-10-06T01:39:49.339868+0000","last_undegraded":"2023-10-06T01:39:49.339868+0000","last_fullsized":"2023-10-06T01:39:49.339868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:58.429853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356192+0000","last_change":"2023-10-06T01:39:49.353291+0000","last_active":"2023-10-06T01:39:49.356192+0000","last_peered":"2023-10-06T01:39:49.356192+0000","last_clean":"2023-10-06T01:39:49.356192+0000","last_became_active":"2023-10-06T01:39:49.352912+0000","last_became_peered":"2023-10-06T01:39:49.352912+0000","last_unstale":"2023-10-06T01:39:49.356192+0000","last_undegraded":"2023-10-06T01:39:49.356192+0000","last_fullsized":"2023-10-06T01:39:49.356192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.229209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.325148+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.324687+0000","last_became_peered":"2023-10-06T01:39:44.324687+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347753+0000","last_change":"2023-10-06T01:39:45.329116+0000","last_active":"2023-10-06T01:39:49.347753+0000","last_peered":"2023-10-06T01:39:49.347753+0000","last_clean":"2023-10-06T01:39:49.347753+0000","last_became_active":"2023-10-06T01:39:45.328839+0000","last_became_peered":"2023-10-06T01:39:45.328839+0000","last_unstale":"2023-10-06T01:39:49.347753+0000","last_undegraded":"2023-10-06T01:39:49.347753+0000","last_fullsized":"2023-10-06T01:39:49.347753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356181+0000","last_change":"2023-10-06T01:39:49.353796+0000","last_active":"2023-10-06T01:39:49.356181+0000","last_peered":"2023-10-06T01:39:49.356181+0000","last_clean":"2023-10-06T01:39:49.356181+0000","last_became_active":"2023-10-06T01:39:49.353138+0000","last_became_peered":"2023-10-06T01:39:49.353138+0000","last_unstale":"2023-10-06T01:39:49.356181+0000","last_undegraded":"2023-10-06T01:39:49.356181+0000","last_fullsized":"2023-10-06T01:39:49.356181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337716+0000","last_change":"2023-10-06T01:39:44.329032+0000","last_active":"2023-10-06T01:39:49.337716+0000","last_peered":"2023-10-06T01:39:49.337716+0000","last_clean":"2023-10-06T01:39:49.337716+0000","last_became_active":"2023-10-06T01:39:44.328775+0000","last_became_peered":"2023-10-06T01:39:44.328775+0000","last_unstale":"2023-10-06T01:39:49.337716+0000","last_undegraded":"2023-10-06T01:39:49.337716+0000","last_fullsized":"2023-10-06T01:39:49.337716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331469+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.331152+0000","last_became_peered":"2023-10-06T01:39:45.331152+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356783+0000","last_change":"2023-10-06T01:39:49.351619+0000","last_active":"2023-10-06T01:39:49.356783+0000","last_peered":"2023-10-06T01:39:49.356783+0000","last_clean":"2023-10-06T01:39:49.356783+0000","last_became_active":"2023-10-06T01:39:49.351271+0000","last_became_peered":"2023-10-06T01:39:49.351271+0000","last_unstale":"2023-10-06T01:39:49.356783+0000","last_undegraded":"2023-10-06T01:39:49.356783+0000","last_fullsized":"2023-10-06T01:39:49.356783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:05.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339967+0000","last_change":"2023-10-06T01:39:44.321209+0000","last_active":"2023-10-06T01:39:49.339967+0000","last_peered":"2023-10-06T01:39:49.339967+0000","last_clean":"2023-10-06T01:39:49.339967+0000","last_became_active":"2023-10-06T01:39:44.320932+0000","last_became_peered":"2023-10-06T01:39:44.320932+0000","last_unstale":"2023-10-06T01:39:49.339967+0000","last_undegraded":"2023-10-06T01:39:49.339967+0000","last_fullsized":"2023-10-06T01:39:49.339967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339741+0000","last_change":"2023-10-06T01:39:45.339859+0000","last_active":"2023-10-06T01:39:49.339741+0000","last_peered":"2023-10-06T01:39:49.339741+0000","last_clean":"2023-10-06T01:39:49.339741+0000","last_became_active":"2023-10-06T01:39:45.339422+0000","last_became_peered":"2023-10-06T01:39:45.339422+0000","last_unstale":"2023-10-06T01:39:49.339741+0000","last_undegraded":"2023-10-06T01:39:49.339741+0000","last_fullsized":"2023-10-06T01:39:49.339741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:36.266399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367304+0000","last_change":"2023-10-06T01:39:49.361654+0000","last_active":"2023-10-06T01:39:49.367304+0000","last_peered":"2023-10-06T01:39:49.367304+0000","last_clean":"2023-10-06T01:39:49.367304+0000","last_became_active":"2023-10-06T01:39:49.361257+0000","last_became_peered":"2023-10-06T01:39:49.361257+0000","last_unstale":"2023-10-06T01:39:49.367304+0000","last_undegraded":"2023-10-06T01:39:49.367304+0000","last_fullsized":"2023-10-06T01:39:49.367304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:13:40.277782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341922+0000","last_change":"2023-10-06T01:39:44.326380+0000","last_active":"2023-10-06T01:39:49.341922+0000","last_peered":"2023-10-06T01:39:49.341922+0000","last_clean":"2023-10-06T01:39:49.341922+0000","last_became_active":"2023-10-06T01:39:44.326059+0000","last_became_peered":"2023-10-06T01:39:44.326059+0000","last_unstale":"2023-10-06T01:39:49.341922+0000","last_undegraded":"2023-10-06T01:39:49.341922+0000","last_fullsized":"2023-10-06T01:39:49.341922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342079+0000","last_change":"2023-10-06T01:39:45.329138+0000","last_active":"2023-10-06T01:39:49.342079+0000","last_peered":"2023-10-06T01:39:49.342079+0000","last_clean":"2023-10-06T01:39:49.342079+0000","last_became_active":"2023-10-06T01:39:45.328861+0000","last_became_peered":"2023-10-06T01:39:45.328861+0000","last_unstale":"2023-10-06T01:39:49.342079+0000","last_undegraded":"2023-10-06T01:39:49.342079+0000","last_fullsized":"2023-10-06T01:39:49.342079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365004+0000","last_change":"2023-10-06T01:39:49.360752+0000","last_active":"2023-10-06T01:39:49.365004+0000","last_peered":"2023-10-06T01:39:49.365004+0000","last_clean":"2023-10-06T01:39:49.365004+0000","last_became_active":"2023-10-06T01:39:49.360376+0000","last_became_peered":"2023-10-06T01:39:49.360376+0000","last_unstale":"2023-10-06T01:39:49.365004+0000","last_undegraded":"2023-10-06T01:39:49.365004+0000","last_fullsized":"2023-10-06T01:39:49.365004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.092106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338022+0000","last_change":"2023-10-06T01:39:44.325482+0000","last_active":"2023-10-06T01:39:49.338022+0000","last_peered":"2023-10-06T01:39:49.338022+0000","last_clean":"2023-10-06T01:39:49.338022+0000","last_became_active":"2023-10-06T01:39:44.325160+0000","last_became_peered":"2023-10-06T01:39:44.325160+0000","last_unstale":"2023-10-06T01:39:49.338022+0000","last_undegraded":"2023-10-06T01:39:49.338022+0000","last_fullsized":"2023-10-06T01:39:49.338022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344360+0000","last_change":"2023-10-06T01:39:45.342331+0000","last_active":"2023-10-06T01:39:49.344360+0000","last_peered":"2023-10-06T01:39:49.344360+0000","last_clean":"2023-10-06T01:39:49.344360+0000","last_became_active":"2023-10-06T01:39:45.342022+0000","last_became_peered":"2023-10-06T01:39:45.342022+0000","last_unstale":"2023-10-06T01:39:49.344360+0000","last_undegraded":"2023-10-06T01:39:49.344360+0000","last_fullsized":"2023-10-06T01:39:49.344360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357018+0000","last_change":"2023-10-06T01:39:49.355423+0000","last_active":"2023-10-06T01:39:49.357018+0000","last_peered":"2023-10-06T01:39:49.357018+0000","last_clean":"2023-10-06T01:39:49.357018+0000","last_became_active":"2023-10-06T01:39:49.355075+0000","last_became_peered":"2023-10-06T01:39:49.355075+0000","last_unstale":"2023-10-06T01:39:49.357018+0000","last_undegraded":"2023-10-06T01:39:49.357018+0000","last_fullsized":"2023-10-06T01:39:49.357018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:02.539610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338012+0000","last_change":"2023-10-06T01:39:44.328632+0000","last_active":"2023-10-06T01:39:49.338012+0000","last_peered":"2023-10-06T01:39:49.338012+0000","last_clean":"2023-10-06T01:39:49.338012+0000","last_became_active":"2023-10-06T01:39:44.326328+0000","last_became_peered":"2023-10-06T01:39:44.326328+0000","last_unstale":"2023-10-06T01:39:49.338012+0000","last_undegraded":"2023-10-06T01:39:49.338012+0000","last_fullsized":"2023-10-06T01:39:49.338012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339913+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:39:49.339913+0000","last_peered":"2023-10-06T01:39:49.339913+0000","last_clean":"2023-10-06T01:39:49.339913+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:39:49.339913+0000","last_undegraded":"2023-10-06T01:39:49.339913+0000","last_fullsized":"2023-10-06T01:39:49.339913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337712+0000","last_change":"2023-10-06T01:39:45.342039+0000","last_active":"2023-10-06T01:39:49.337712+0000","last_peered":"2023-10-06T01:39:49.337712+0000","last_clean":"2023-10-06T01:39:49.337712+0000","last_became_active":"2023-10-06T01:39:45.341746+0000","last_became_peered":"2023-10-06T01:39:45.341746+0000","last_unstale":"2023-10-06T01:39:49.337712+0000","last_undegraded":"2023-10-06T01:39:49.337712+0000","last_fullsized":"2023-10-06T01:39:49.337712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354087+0000","last_change":"2023-10-06T01:39:49.352264+0000","last_active":"2023-10-06T01:39:49.354087+0000","last_peered":"2023-10-06T01:39:49.354087+0000","last_clean":"2023-10-06T01:39:49.354087+0000","last_became_active":"2023-10-06T01:39:49.351974+0000","last_became_peered":"2023-10-06T01:39:49.351974+0000","last_unstale":"2023-10-06T01:39:49.354087+0000","last_undegraded":"2023-10-06T01:39:49.354087+0000","last_fullsized":"2023-10-06T01:39:49.354087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:35.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337764+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:39:49.337764+0000","last_peered":"2023-10-06T01:39:49.337764+0000","last_clean":"2023-10-06T01:39:49.337764+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:39:49.337764+0000","last_undegraded":"2023-10-06T01:39:49.337764+0000","last_fullsized":"2023-10-06T01:39:49.337764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.395905+0000","last_change":"2023-10-06T01:39:44.324833+0000","last_active":"2023-10-06T01:39:49.395905+0000","last_peered":"2023-10-06T01:39:49.395905+0000","last_clean":"2023-10-06T01:39:49.395905+0000","last_became_active":"2023-10-06T01:39:44.324554+0000","last_became_peered":"2023-10-06T01:39:44.324554+0000","last_unstale":"2023-10-06T01:39:49.395905+0000","last_undegraded":"2023-10-06T01:39:49.395905+0000","last_fullsized":"2023-10-06T01:39:49.395905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:53.581079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341933+0000","last_change":"2023-10-06T01:39:45.331099+0000","last_active":"2023-10-06T01:39:49.341933+0000","last_peered":"2023-10-06T01:39:49.341933+0000","last_clean":"2023-10-06T01:39:49.341933+0000","last_became_active":"2023-10-06T01:39:45.330844+0000","last_became_peered":"2023-10-06T01:39:45.330844+0000","last_unstale":"2023-10-06T01:39:49.341933+0000","last_undegraded":"2023-10-06T01:39:49.341933+0000","last_fullsized":"2023-10-06T01:39:49.341933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367282+0000","last_change":"2023-10-06T01:39:49.360839+0000","last_active":"2023-10-06T01:39:49.367282+0000","last_peered":"2023-10-06T01:39:49.367282+0000","last_clean":"2023-10-06T01:39:49.367282+0000","last_became_active":"2023-10-06T01:39:49.360447+0000","last_became_peered":"2023-10-06T01:39:49.360447+0000","last_unstale":"2023-10-06T01:39:49.367282+0000","last_undegraded":"2023-10-06T01:39:49.367282+0000","last_fullsized":"2023-10-06T01:39:49.367282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342223+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:39:49.342223+0000","last_peered":"2023-10-06T01:39:49.342223+0000","last_clean":"2023-10-06T01:39:49.342223+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:39:49.342223+0000","last_undegraded":"2023-10-06T01:39:49.342223+0000","last_fullsized":"2023-10-06T01:39:49.342223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350783+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:39:49.350783+0000","last_peered":"2023-10-06T01:39:49.350783+0000","last_clean":"2023-10-06T01:39:49.350783+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:39:49.350783+0000","last_undegraded":"2023-10-06T01:39:49.350783+0000","last_fullsized":"2023-10-06T01:39:49.350783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346294+0000","last_change":"2023-10-06T01:39:44.332942+0000","last_active":"2023-10-06T01:39:49.346294+0000","last_peered":"2023-10-06T01:39:49.346294+0000","last_clean":"2023-10-06T01:39:49.346294+0000","last_became_active":"2023-10-06T01:39:44.332767+0000","last_became_peered":"2023-10-06T01:39:44.332767+0000","last_unstale":"2023-10-06T01:39:49.346294+0000","last_undegraded":"2023-10-06T01:39:49.346294+0000","last_fullsized":"2023-10-06T01:39:49.346294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351451+0000","last_change":"2023-10-06T01:39:45.331974+0000","last_active":"2023-10-06T01:39:49.351451+0000","last_peered":"2023-10-06T01:39:49.351451+0000","last_clean":"2023-10-06T01:39:49.351451+0000","last_became_active":"2023-10-06T01:39:45.331706+0000","last_became_peered":"2023-10-06T01:39:45.331706+0000","last_unstale":"2023-10-06T01:39:49.351451+0000","last_undegraded":"2023-10-06T01:39:49.351451+0000","last_fullsized":"2023-10-06T01:39:49.351451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357328+0000","last_change":"2023-10-06T01:39:49.356514+0000","last_active":"2023-10-06T01:39:49.357328+0000","last_peered":"2023-10-06T01:39:49.357328+0000","last_clean":"2023-10-06T01:39:49.357328+0000","last_became_active":"2023-10-06T01:39:49.356017+0000","last_became_peered":"2023-10-06T01:39:49.356017+0000","last_unstale":"2023-10-06T01:39:49.357328+0000","last_undegraded":"2023-10-06T01:39:49.357328+0000","last_fullsized":"2023-10-06T01:39:49.357328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:28.616739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350977+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:39:49.350977+0000","last_peered":"2023-10-06T01:39:49.350977+0000","last_clean":"2023-10-06T01:39:49.350977+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:39:49.350977+0000","last_undegraded":"2023-10-06T01:39:49.350977+0000","last_fullsized":"2023-10-06T01:39:49.350977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345374+0000","last_change":"2023-10-06T01:39:44.324538+0000","last_active":"2023-10-06T01:39:49.345374+0000","last_peered":"2023-10-06T01:39:49.345374+0000","last_clean":"2023-10-06T01:39:49.345374+0000","last_became_active":"2023-10-06T01:39:44.324239+0000","last_became_peered":"2023-10-06T01:39:44.324239+0000","last_unstale":"2023-10-06T01:39:49.345374+0000","last_undegraded":"2023-10-06T01:39:49.345374+0000","last_fullsized":"2023-10-06T01:39:49.345374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344282+0000","last_change":"2023-10-06T01:39:45.339432+0000","last_active":"2023-10-06T01:39:49.344282+0000","last_peered":"2023-10-06T01:39:49.344282+0000","last_clean":"2023-10-06T01:39:49.344282+0000","last_became_active":"2023-10-06T01:39:45.339191+0000","last_became_peered":"2023-10-06T01:39:45.339191+0000","last_unstale":"2023-10-06T01:39:49.344282+0000","last_undegraded":"2023-10-06T01:39:49.344282+0000","last_fullsized":"2023-10-06T01:39:49.344282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363746+0000","last_change":"2023-10-06T01:39:49.361536+0000","last_active":"2023-10-06T01:39:49.363746+0000","last_peered":"2023-10-06T01:39:49.363746+0000","last_clean":"2023-10-06T01:39:49.363746+0000","last_became_active":"2023-10-06T01:39:49.361162+0000","last_became_peered":"2023-10-06T01:39:49.361162+0000","last_unstale":"2023-10-06T01:39:49.363746+0000","last_undegraded":"2023-10-06T01:39:49.363746+0000","last_fullsized":"2023-10-06T01:39:49.363746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350576+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:39:49.350576+0000","last_peered":"2023-10-06T01:39:49.350576+0000","last_clean":"2023-10-06T01:39:49.350576+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:39:49.350576+0000","last_undegraded":"2023-10-06T01:39:49.350576+0000","last_fullsized":"2023-10-06T01:39:49.350576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337530+0000","last_change":"2023-10-06T01:39:44.328264+0000","last_active":"2023-10-06T01:39:49.337530+0000","last_peered":"2023-10-06T01:39:49.337530+0000","last_clean":"2023-10-06T01:39:49.337530+0000","last_became_active":"2023-10-06T01:39:44.327969+0000","last_became_peered":"2023-10-06T01:39:44.327969+0000","last_unstale":"2023-10-06T01:39:49.337530+0000","last_undegraded":"2023-10-06T01:39:49.337530+0000","last_fullsized":"2023-10-06T01:39:49.337530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340324+0000","last_change":"2023-10-06T01:39:45.333753+0000","last_active":"2023-10-06T01:39:49.340324+0000","last_peered":"2023-10-06T01:39:49.340324+0000","last_clean":"2023-10-06T01:39:49.340324+0000","last_became_active":"2023-10-06T01:39:45.333416+0000","last_became_peered":"2023-10-06T01:39:45.333416+0000","last_unstale":"2023-10-06T01:39:49.340324+0000","last_undegraded":"2023-10-06T01:39:49.340324+0000","last_fullsized":"2023-10-06T01:39:49.340324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361707+0000","last_change":"2023-10-06T01:39:49.360037+0000","last_active":"2023-10-06T01:39:49.361707+0000","last_peered":"2023-10-06T01:39:49.361707+0000","last_clean":"2023-10-06T01:39:49.361707+0000","last_became_active":"2023-10-06T01:39:49.359697+0000","last_became_peered":"2023-10-06T01:39:49.359697+0000","last_unstale":"2023-10-06T01:39:49.361707+0000","last_undegraded":"2023-10-06T01:39:49.361707+0000","last_fullsized":"2023-10-06T01:39:49.361707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349757+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:39:49.349757+0000","last_peered":"2023-10-06T01:39:49.349757+0000","last_clean":"2023-10-06T01:39:49.349757+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:39:49.349757+0000","last_undegraded":"2023-10-06T01:39:49.349757+0000","last_fullsized":"2023-10-06T01:39:49.349757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:44.329991+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:44.329756+0000","last_became_peered":"2023-10-06T01:39:44.329756+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343619+0000","last_change":"2023-10-06T01:39:45.336684+0000","last_active":"2023-10-06T01:39:49.343619+0000","last_peered":"2023-10-06T01:39:49.343619+0000","last_clean":"2023-10-06T01:39:49.343619+0000","last_became_active":"2023-10-06T01:39:45.336427+0000","last_became_peered":"2023-10-06T01:39:45.336427+0000","last_unstale":"2023-10-06T01:39:49.343619+0000","last_undegraded":"2023-10-06T01:39:49.343619+0000","last_fullsized":"2023-10-06T01:39:49.343619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359866+0000","last_change":"2023-10-06T01:39:49.358138+0000","last_active":"2023-10-06T01:39:49.359866+0000","last_peered":"2023-10-06T01:39:49.359866+0000","last_clean":"2023-10-06T01:39:49.359866+0000","last_became_active":"2023-10-06T01:39:49.356144+0000","last_became_peered":"2023-10-06T01:39:49.356144+0000","last_unstale":"2023-10-06T01:39:49.359866+0000","last_undegraded":"2023-10-06T01:39:49.359866+0000","last_fullsized":"2023-10-06T01:39:49.359866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:02.622952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341234+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:39:49.341234+0000","last_peered":"2023-10-06T01:39:49.341234+0000","last_clean":"2023-10-06T01:39:49.341234+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:39:49.341234+0000","last_undegraded":"2023-10-06T01:39:49.341234+0000","last_fullsized":"2023-10-06T01:39:49.341234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343734+0000","last_change":"2023-10-06T01:39:44.327413+0000","last_active":"2023-10-06T01:39:49.343734+0000","last_peered":"2023-10-06T01:39:49.343734+0000","last_clean":"2023-10-06T01:39:49.343734+0000","last_became_active":"2023-10-06T01:39:44.327177+0000","last_became_peered":"2023-10-06T01:39:44.327177+0000","last_unstale":"2023-10-06T01:39:49.343734+0000","last_undegraded":"2023-10-06T01:39:49.343734+0000","last_fullsized":"2023-10-06T01:39:49.343734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:17.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347454+0000","last_change":"2023-10-06T01:39:45.334858+0000","last_active":"2023-10-06T01:39:49.347454+0000","last_peered":"2023-10-06T01:39:49.347454+0000","last_clean":"2023-10-06T01:39:49.347454+0000","last_became_active":"2023-10-06T01:39:45.334586+0000","last_became_peered":"2023-10-06T01:39:45.334586+0000","last_unstale":"2023-10-06T01:39:49.347454+0000","last_undegraded":"2023-10-06T01:39:49.347454+0000","last_fullsized":"2023-10-06T01:39:49.347454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:10.350398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.366854+0000","last_change":"2023-10-06T01:39:49.365711+0000","last_active":"2023-10-06T01:39:49.366854+0000","last_peered":"2023-10-06T01:39:49.366854+0000","last_clean":"2023-10-06T01:39:49.366854+0000","last_became_active":"2023-10-06T01:39:49.365357+0000","last_became_peered":"2023-10-06T01:39:49.365357+0000","last_unstale":"2023-10-06T01:39:49.366854+0000","last_undegraded":"2023-10-06T01:39:49.366854+0000","last_fullsized":"2023-10-06T01:39:49.366854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339829+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:39:49.339829+0000","last_peered":"2023-10-06T01:39:49.339829+0000","last_clean":"2023-10-06T01:39:49.339829+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:39:49.339829+0000","last_undegraded":"2023-10-06T01:39:49.339829+0000","last_fullsized":"2023-10-06T01:39:49.339829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350343+0000","last_change":"2023-10-06T01:39:44.327795+0000","last_active":"2023-10-06T01:39:49.350343+0000","last_peered":"2023-10-06T01:39:49.350343+0000","last_clean":"2023-10-06T01:39:49.350343+0000","last_became_active":"2023-10-06T01:39:44.326791+0000","last_became_peered":"2023-10-06T01:39:44.326791+0000","last_unstale":"2023-10-06T01:39:49.350343+0000","last_undegraded":"2023-10-06T01:39:49.350343+0000","last_fullsized":"2023-10-06T01:39:49.350343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342538+0000","last_change":"2023-10-06T01:39:45.341418+0000","last_active":"2023-10-06T01:39:49.342538+0000","last_peered":"2023-10-06T01:39:49.342538+0000","last_clean":"2023-10-06T01:39:49.342538+0000","last_became_active":"2023-10-06T01:39:45.341053+0000","last_became_peered":"2023-10-06T01:39:45.341053+0000","last_unstale":"2023-10-06T01:39:49.342538+0000","last_undegraded":"2023-10-06T01:39:49.342538+0000","last_fullsized":"2023-10-06T01:39:49.342538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356900+0000","last_change":"2023-10-06T01:39:49.355119+0000","last_active":"2023-10-06T01:39:49.356900+0000","last_peered":"2023-10-06T01:39:49.356900+0000","last_clean":"2023-10-06T01:39:49.356900+0000","last_became_active":"2023-10-06T01:39:49.354770+0000","last_became_peered":"2023-10-06T01:39:49.354770+0000","last_unstale":"2023-10-06T01:39:49.356900+0000","last_undegraded":"2023-10-06T01:39:49.356900+0000","last_fullsized":"2023-10-06T01:39:49.356900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:40.295194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347370+0000","last_change":"2023-10-06T01:39:44.333929+0000","last_active":"2023-10-06T01:39:49.347370+0000","last_peered":"2023-10-06T01:39:49.347370+0000","last_clean":"2023-10-06T01:39:49.347370+0000","last_became_active":"2023-10-06T01:39:44.333659+0000","last_became_peered":"2023-10-06T01:39:44.333659+0000","last_unstale":"2023-10-06T01:39:49.347370+0000","last_undegraded":"2023-10-06T01:39:49.347370+0000","last_fullsized":"2023-10-06T01:39:49.347370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346317+0000","last_change":"2023-10-06T01:39:45.331228+0000","last_active":"2023-10-06T01:39:49.346317+0000","last_peered":"2023-10-06T01:39:49.346317+0000","last_clean":"2023-10-06T01:39:49.346317+0000","last_became_active":"2023-10-06T01:39:45.330982+0000","last_became_peered":"2023-10-06T01:39:45.330982+0000","last_unstale":"2023-10-06T01:39:49.346317+0000","last_undegraded":"2023-10-06T01:39:49.346317+0000","last_fullsized":"2023-10-06T01:39:49.346317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.923135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363774+0000","last_change":"2023-10-06T01:39:49.362572+0000","last_active":"2023-10-06T01:39:49.363774+0000","last_peered":"2023-10-06T01:39:49.363774+0000","last_clean":"2023-10-06T01:39:49.363774+0000","last_became_active":"2023-10-06T01:39:49.362200+0000","last_became_peered":"2023-10-06T01:39:49.362200+0000","last_unstale":"2023-10-06T01:39:49.363774+0000","last_undegraded":"2023-10-06T01:39:49.363774+0000","last_fullsized":"2023-10-06T01:39:49.363774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349796+0000","last_change":"2023-10-06T01:39:44.332451+0000","last_active":"2023-10-06T01:39:49.349796+0000","last_peered":"2023-10-06T01:39:49.349796+0000","last_clean":"2023-10-06T01:39:49.349796+0000","last_became_active":"2023-10-06T01:39:44.332143+0000","last_became_peered":"2023-10-06T01:39:44.332143+0000","last_unstale":"2023-10-06T01:39:49.349796+0000","last_undegraded":"2023-10-06T01:39:49.349796+0000","last_fullsized":"2023-10-06T01:39:49.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331260+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.330980+0000","last_became_peered":"2023-10-06T01:39:45.330980+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368960+0000","last_change":"2023-10-06T01:39:49.361125+0000","last_active":"2023-10-06T01:39:49.368960+0000","last_peered":"2023-10-06T01:39:49.368960+0000","last_clean":"2023-10-06T01:39:49.368960+0000","last_became_active":"2023-10-06T01:39:49.354967+0000","last_became_peered":"2023-10-06T01:39:49.354967+0000","last_unstale":"2023-10-06T01:39:49.368960+0000","last_undegraded":"2023-10-06T01:39:49.368960+0000","last_fullsized":"2023-10-06T01:39:49.368960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:41.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.329353+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.329098+0000","last_became_peered":"2023-10-06T01:39:44.329098+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340491+0000","last_change":"2023-10-06T01:39:45.330780+0000","last_active":"2023-10-06T01:39:49.340491+0000","last_peered":"2023-10-06T01:39:49.340491+0000","last_clean":"2023-10-06T01:39:49.340491+0000","last_became_active":"2023-10-06T01:39:45.330484+0000","last_became_peered":"2023-10-06T01:39:45.330484+0000","last_unstale":"2023-10-06T01:39:49.340491+0000","last_undegraded":"2023-10-06T01:39:49.340491+0000","last_fullsized":"2023-10-06T01:39:49.340491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354659+0000","last_change":"2023-10-06T01:39:49.351706+0000","last_active":"2023-10-06T01:39:49.354659+0000","last_peered":"2023-10-06T01:39:49.354659+0000","last_clean":"2023-10-06T01:39:49.354659+0000","last_became_active":"2023-10-06T01:39:49.350630+0000","last_became_peered":"2023-10-06T01:39:49.350630+0000","last_unstale":"2023-10-06T01:39:49.354659+0000","last_undegraded":"2023-10-06T01:39:49.354659+0000","last_fullsized":"2023-10-06T01:39:49.354659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:57.336637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337644+0000","last_change":"2023-10-06T01:39:44.321387+0000","last_active":"2023-10-06T01:39:49.337644+0000","last_peered":"2023-10-06T01:39:49.337644+0000","last_clean":"2023-10-06T01:39:49.337644+0000","last_became_active":"2023-10-06T01:39:44.321101+0000","last_became_peered":"2023-10-06T01:39:44.321101+0000","last_unstale":"2023-10-06T01:39:49.337644+0000","last_undegraded":"2023-10-06T01:39:49.337644+0000","last_fullsized":"2023-10-06T01:39:49.337644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337678+0000","last_change":"2023-10-06T01:39:45.333689+0000","last_active":"2023-10-06T01:39:49.337678+0000","last_peered":"2023-10-06T01:39:49.337678+0000","last_clean":"2023-10-06T01:39:49.337678+0000","last_became_active":"2023-10-06T01:39:45.333444+0000","last_became_peered":"2023-10-06T01:39:45.333444+0000","last_unstale":"2023-10-06T01:39:49.337678+0000","last_undegraded":"2023-10-06T01:39:49.337678+0000","last_fullsized":"2023-10-06T01:39:49.337678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.362749+0000","last_change":"2023-10-06T01:39:49.360875+0000","last_active":"2023-10-06T01:39:49.362749+0000","last_peered":"2023-10-06T01:39:49.362749+0000","last_clean":"2023-10-06T01:39:49.362749+0000","last_became_active":"2023-10-06T01:39:49.360515+0000","last_became_peered":"2023-10-06T01:39:49.360515+0000","last_unstale":"2023-10-06T01:39:49.362749+0000","last_undegraded":"2023-10-06T01:39:49.362749+0000","last_fullsized":"2023-10-06T01:39:49.362749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351554+0000","last_change":"2023-10-06T01:39:44.326433+0000","last_active":"2023-10-06T01:39:49.351554+0000","last_peered":"2023-10-06T01:39:49.351554+0000","last_clean":"2023-10-06T01:39:49.351554+0000","last_became_active":"2023-10-06T01:39:44.326167+0000","last_became_peered":"2023-10-06T01:39:44.326167+0000","last_unstale":"2023-10-06T01:39:49.351554+0000","last_undegraded":"2023-10-06T01:39:49.351554+0000","last_fullsized":"2023-10-06T01:39:49.351554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342356+0000","last_change":"2023-10-06T01:39:45.330018+0000","last_active":"2023-10-06T01:39:49.342356+0000","last_peered":"2023-10-06T01:39:49.342356+0000","last_clean":"2023-10-06T01:39:49.342356+0000","last_became_active":"2023-10-06T01:39:45.329556+0000","last_became_peered":"2023-10-06T01:39:45.329556+0000","last_unstale":"2023-10-06T01:39:49.342356+0000","last_undegraded":"2023-10-06T01:39:49.342356+0000","last_fullsized":"2023-10-06T01:39:49.342356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343235+0000","last_change":"2023-10-06T01:39:45.333252+0000","last_active":"2023-10-06T01:39:49.343235+0000","last_peered":"2023-10-06T01:39:49.343235+0000","last_clean":"2023-10-06T01:39:49.343235+0000","last_became_active":"2023-10-06T01:39:45.332899+0000","last_became_peered":"2023-10-06T01:39:45.332899+0000","last_unstale":"2023-10-06T01:39:49.343235+0000","last_undegraded":"2023-10-06T01:39:49.343235+0000","last_fullsized":"2023-10-06T01:39:49.343235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:37.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337491+0000","last_change":"2023-10-06T01:39:45.342104+0000","last_active":"2023-10-06T01:39:49.337491+0000","last_peered":"2023-10-06T01:39:49.337491+0000","last_clean":"2023-10-06T01:39:49.337491+0000","last_became_active":"2023-10-06T01:39:45.341825+0000","last_became_peered":"2023-10-06T01:39:45.341825+0000","last_unstale":"2023-10-06T01:39:49.337491+0000","last_undegraded":"2023-10-06T01:39:49.337491+0000","last_fullsized":"2023-10-06T01:39:49.337491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340694+0000","last_change":"2023-10-06T01:39:45.329243+0000","last_active":"2023-10-06T01:39:49.340694+0000","last_peered":"2023-10-06T01:39:49.340694+0000","last_clean":"2023-10-06T01:39:49.340694+0000","last_became_active":"2023-10-06T01:39:45.328438+0000","last_became_peered":"2023-10-06T01:39:45.328438+0000","last_unstale":"2023-10-06T01:39:49.340694+0000","last_undegraded":"2023-10-06T01:39:49.340694+0000","last_fullsized":"2023-10-06T01:39:49.340694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337590+0000","last_change":"2023-10-06T01:39:45.330664+0000","last_active":"2023-10-06T01:39:49.337590+0000","last_peered":"2023-10-06T01:39:49.337590+0000","last_clean":"2023-10-06T01:39:49.337590+0000","last_became_active":"2023-10-06T01:39:45.330394+0000","last_became_peered":"2023-10-06T01:39:45.330394+0000","last_unstale":"2023-10-06T01:39:49.337590+0000","last_undegraded":"2023-10-06T01:39:49.337590+0000","last_fullsized":"2023-10-06T01:39:49.337590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343700+0000","last_change":"2023-10-06T01:39:45.338933+0000","last_active":"2023-10-06T01:39:49.343700+0000","last_peered":"2023-10-06T01:39:49.343700+0000","last_clean":"2023-10-06T01:39:49.343700+0000","last_became_active":"2023-10-06T01:39:45.338515+0000","last_became_peered":"2023-10-06T01:39:45.338515+0000","last_unstale":"2023-10-06T01:39:49.343700+0000","last_undegraded":"2023-10-06T01:39:49.343700+0000","last_fullsized":"2023-10-06T01:39:49.343700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":6791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":696558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:39:59.063 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:39:59.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.204+0000 7f05f9657700 1 Processor -- start 2023-10-06T01:39:59.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.204+0000 7f05f9657700 1 -- start start 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f9657700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f41645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f9657700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f40b0de0 con 0x7f05f40b2620 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f9657700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f40cbfe0 con 0x7f05f40b29f0 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f9657700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40cb3d0 con 0x7f05f40cd480 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f2ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f05f40b2620 0x7f05f40b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36388/0 (socket says 172.21.15.98:36388) 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f2ffd700 1 -- 172.21.15.98:0/773348141 learned_addr learned my addr 172.21.15.98:0/773348141 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:59.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.205+0000 7f05f37fe700 1 --2- 172.21.15.98:0/773348141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f41645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413479133 0 0) 0x7f05e4001610 con 0x7f05f40b2620 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e0003570 con 0x7f05f40b2620 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941703759 0 0) 0x7f05f40cbfe0 con 0x7f05f40b29f0 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e4001610 con 0x7f05f40b29f0 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f37fe700 1 -- 172.21.15.98:0/773348141 >> v1:172.21.15.184:6789/0 conn(0x7f05f40b29f0 legacy=0x7f05f4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f37fe700 1 -- 172.21.15.98:0/773348141 >> v1:172.21.15.98:6789/0 conn(0x7f05f40b2620 legacy=0x7f05f40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f37fe700 1 -- 172.21.15.98:0/773348141 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40cb6b0 con 0x7f05f40cd480 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f37fe700 1 --2- 172.21.15.98:0/773348141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f41645e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009910 tx=0x7f05ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9ca1b7020c4f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011630 con 0x7f05f40cd480 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec011790 con 0x7f05f40cd480 2023-10-06T01:39:59.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.206+0000 7f05f1ffb700 1 -- 172.21.15.98:0/773348141 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011b30 con 0x7f05f40cd480 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 -- 172.21.15.98:0/773348141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 msgr2=0x7f05f41645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 --2- 172.21.15.98:0/773348141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f41645e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009910 tx=0x7f05ec0091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 -- 172.21.15.98:0/773348141 shutdown_connections 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 --2- 172.21.15.98:0/773348141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f41645e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 -- 172.21.15.98:0/773348141 >> 172.21.15.98:0/773348141 conn(0x7f05f406ed20 msgr2=0x7f05f406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 -- 172.21.15.98:0/773348141 shutdown_connections 2023-10-06T01:39:59.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.207+0000 7f05f9657700 1 -- 172.21.15.98:0/773348141 wait complete. 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.208+0000 7f05f9657700 1 Processor -- start 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.208+0000 7f05f9657700 1 -- start start 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f40b2620 0x7f05f414b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f4149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f414b750 con 0x7f05f40b29f0 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f414dde0 con 0x7f05f40b2620 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f414e0c0 con 0x7f05f40cd480 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34056/0 (socket says 172.21.15.98:34056) 2023-10-06T01:39:59.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 -- 172.21.15.98:0/2109728781 learned_addr learned my addr 172.21.15.98:0/2109728781 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:59.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f2ffd700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f40b2620 0x7f05f414b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f37fe700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f4149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 -- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 msgr2=0x7f05f4149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f4149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 -- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f40b2620 msgr2=0x7f05f414b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f40b2620 0x7f05f414b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 -- 172.21.15.98:0/2109728781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009420 con 0x7f05f40b29f0 2023-10-06T01:39:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f27fc700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c790 tx=0x7f05dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010510 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc010670 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f414e3a0 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010a10 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.209+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f415ef90 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.210+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f415f1b0 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.211+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05dc01b400 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.211+0000 7f05eaffd700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 0x7f05d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.211+0000 7f05f2ffd700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 0x7f05d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.211+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f05dc084750 con 0x7f05f40b29f0 2023-10-06T01:39:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.212+0000 7f05f2ffd700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 0x7f05d40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05e4001870 tx=0x7f05e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.215+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05dc051570 con 0x7f05f40b29f0 2023-10-06T01:39:59.394 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.393+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05f4064ad0 con 0x7f05d4066f30 2023-10-06T01:39:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.402+0000 7f05eaffd700 1 -- 172.21.15.98:0/2109728781 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367627 (secure 0 0 0) 0x7f05f4064ad0 con 0x7f05d4066f30 2023-10-06T01:39:59.408 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 msgr2=0x7f05d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 0x7f05d40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05e4001870 tx=0x7f05e4009040 comp rx=0 tx=0).stop 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 msgr2=0x7f05f415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c790 tx=0x7f05dc007e10 comp rx=0 tx=0).stop 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 shutdown_connections 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05d4066f30 0x7f05d40693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05f40cd480 0x7f05f4149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f40b2620 0x7f05f414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 --2- 172.21.15.98:0/2109728781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05f40b29f0 0x7f05f415f4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 >> 172.21.15.98:0/2109728781 conn(0x7f05f406ed20 msgr2=0x7f05f40cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 shutdown_connections 2023-10-06T01:39:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.406+0000 7f05f9657700 1 -- 172.21.15.98:0/2109728781 wait complete. 2023-10-06T01:39:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:39:59.420 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T01:39:57.787301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":327,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2021678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.451112"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346761+0000","last_change":"2023-10-06T01:39:45.335803+0000","last_active":"2023-10-06T01:39:49.346761+0000","last_peered":"2023-10-06T01:39:49.346761+0000","last_clean":"2023-10-06T01:39:49.346761+0000","last_became_active":"2023-10-06T01:39:45.335420+0000","last_became_peered":"2023-10-06T01:39:45.335420+0000","last_unstale":"2023-10-06T01:39:49.346761+0000","last_undegraded":"2023-10-06T01:39:49.346761+0000","last_fullsized":"2023-10-06T01:39:49.346761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:58.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353326+0000","last_change":"2023-10-06T01:39:45.336392+0000","last_active":"2023-10-06T01:39:49.353326+0000","last_peered":"2023-10-06T01:39:49.353326+0000","last_clean":"2023-10-06T01:39:49.353326+0000","last_became_active":"2023-10-06T01:39:45.336114+0000","last_became_peered":"2023-10-06T01:39:45.336114+0000","last_unstale":"2023-10-06T01:39:49.353326+0000","last_undegraded":"2023-10-06T01:39:49.353326+0000","last_fullsized":"2023-10-06T01:39:49.353326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340003+0000","last_change":"2023-10-06T01:39:45.330889+0000","last_active":"2023-10-06T01:39:49.340003+0000","last_peered":"2023-10-06T01:39:49.340003+0000","last_clean":"2023-10-06T01:39:49.340003+0000","last_became_active":"2023-10-06T01:39:45.330584+0000","last_became_peered":"2023-10-06T01:39:45.330584+0000","last_unstale":"2023-10-06T01:39:49.340003+0000","last_undegraded":"2023-10-06T01:39:49.340003+0000","last_fullsized":"2023-10-06T01:39:49.340003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:55.095137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340381+0000","last_change":"2023-10-06T01:39:45.327678+0000","last_active":"2023-10-06T01:39:49.340381+0000","last_peered":"2023-10-06T01:39:49.340381+0000","last_clean":"2023-10-06T01:39:49.340381+0000","last_became_active":"2023-10-06T01:39:45.327329+0000","last_became_peered":"2023-10-06T01:39:45.327329+0000","last_unstale":"2023-10-06T01:39:49.340381+0000","last_undegraded":"2023-10-06T01:39:49.340381+0000","last_fullsized":"2023-10-06T01:39:49.340381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:13.570152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347538+0000","last_change":"2023-10-06T01:39:45.342805+0000","last_active":"2023-10-06T01:39:49.347538+0000","last_peered":"2023-10-06T01:39:49.347538+0000","last_clean":"2023-10-06T01:39:49.347538+0000","last_became_active":"2023-10-06T01:39:45.342510+0000","last_became_peered":"2023-10-06T01:39:45.342510+0000","last_unstale":"2023-10-06T01:39:49.347538+0000","last_undegraded":"2023-10-06T01:39:49.347538+0000","last_fullsized":"2023-10-06T01:39:49.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340791+0000","last_change":"2023-10-06T01:39:45.328521+0000","last_active":"2023-10-06T01:39:49.340791+0000","last_peered":"2023-10-06T01:39:49.340791+0000","last_clean":"2023-10-06T01:39:49.340791+0000","last_became_active":"2023-10-06T01:39:45.327249+0000","last_became_peered":"2023-10-06T01:39:45.327249+0000","last_unstale":"2023-10-06T01:39:49.340791+0000","last_undegraded":"2023-10-06T01:39:49.340791+0000","last_fullsized":"2023-10-06T01:39:49.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.886175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350835+0000","last_change":"2023-10-06T01:39:45.336408+0000","last_active":"2023-10-06T01:39:49.350835+0000","last_peered":"2023-10-06T01:39:49.350835+0000","last_clean":"2023-10-06T01:39:49.350835+0000","last_became_active":"2023-10-06T01:39:45.336066+0000","last_became_peered":"2023-10-06T01:39:45.336066+0000","last_unstale":"2023-10-06T01:39:49.350835+0000","last_undegraded":"2023-10-06T01:39:49.350835+0000","last_fullsized":"2023-10-06T01:39:49.350835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350056+0000","last_change":"2023-10-06T01:39:45.331003+0000","last_active":"2023-10-06T01:39:49.350056+0000","last_peered":"2023-10-06T01:39:49.350056+0000","last_clean":"2023-10-06T01:39:49.350056+0000","last_became_active":"2023-10-06T01:39:45.330563+0000","last_became_peered":"2023-10-06T01:39:45.330563+0000","last_unstale":"2023-10-06T01:39:49.350056+0000","last_undegraded":"2023-10-06T01:39:49.350056+0000","last_fullsized":"2023-10-06T01:39:49.350056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351292+0000","last_change":"2023-10-06T01:39:45.344173+0000","last_active":"2023-10-06T01:39:49.351292+0000","last_peered":"2023-10-06T01:39:49.351292+0000","last_clean":"2023-10-06T01:39:49.351292+0000","last_became_active":"2023-10-06T01:39:45.343891+0000","last_became_peered":"2023-10-06T01:39:45.343891+0000","last_unstale":"2023-10-06T01:39:49.351292+0000","last_undegraded":"2023-10-06T01:39:49.351292+0000","last_fullsized":"2023-10-06T01:39:49.351292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353439+0000","last_change":"2023-10-06T01:39:45.332773+0000","last_active":"2023-10-06T01:39:49.353439+0000","last_peered":"2023-10-06T01:39:49.353439+0000","last_clean":"2023-10-06T01:39:49.353439+0000","last_became_active":"2023-10-06T01:39:45.332504+0000","last_became_peered":"2023-10-06T01:39:45.332504+0000","last_unstale":"2023-10-06T01:39:49.353439+0000","last_undegraded":"2023-10-06T01:39:49.353439+0000","last_fullsized":"2023-10-06T01:39:49.353439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340177+0000","last_change":"2023-10-06T01:39:45.335528+0000","last_active":"2023-10-06T01:39:49.340177+0000","last_peered":"2023-10-06T01:39:49.340177+0000","last_clean":"2023-10-06T01:39:49.340177+0000","last_became_active":"2023-10-06T01:39:45.335296+0000","last_became_peered":"2023-10-06T01:39:45.335296+0000","last_unstale":"2023-10-06T01:39:49.340177+0000","last_undegraded":"2023-10-06T01:39:49.340177+0000","last_fullsized":"2023-10-06T01:39:49.340177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:09.004117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346177+0000","last_change":"2023-10-06T01:39:45.326855+0000","last_active":"2023-10-06T01:39:49.346177+0000","last_peered":"2023-10-06T01:39:49.346177+0000","last_clean":"2023-10-06T01:39:49.346177+0000","last_became_active":"2023-10-06T01:39:45.326260+0000","last_became_peered":"2023-10-06T01:39:45.326260+0000","last_unstale":"2023-10-06T01:39:49.346177+0000","last_undegraded":"2023-10-06T01:39:49.346177+0000","last_fullsized":"2023-10-06T01:39:49.346177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:26.407500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345996+0000","last_change":"2023-10-06T01:39:45.346287+0000","last_active":"2023-10-06T01:39:49.345996+0000","last_peered":"2023-10-06T01:39:49.345996+0000","last_clean":"2023-10-06T01:39:49.345996+0000","last_became_active":"2023-10-06T01:39:45.346017+0000","last_became_peered":"2023-10-06T01:39:45.346017+0000","last_unstale":"2023-10-06T01:39:49.345996+0000","last_undegraded":"2023-10-06T01:39:49.345996+0000","last_fullsized":"2023-10-06T01:39:49.345996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:37.090919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350016+0000","last_change":"2023-10-06T01:39:45.337191+0000","last_active":"2023-10-06T01:39:49.350016+0000","last_peered":"2023-10-06T01:39:49.350016+0000","last_clean":"2023-10-06T01:39:49.350016+0000","last_became_active":"2023-10-06T01:39:45.336905+0000","last_became_peered":"2023-10-06T01:39:45.336905+0000","last_unstale":"2023-10-06T01:39:49.350016+0000","last_undegraded":"2023-10-06T01:39:49.350016+0000","last_fullsized":"2023-10-06T01:39:49.350016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350613+0000","last_change":"2023-10-06T01:39:45.336653+0000","last_active":"2023-10-06T01:39:49.350613+0000","last_peered":"2023-10-06T01:39:49.350613+0000","last_clean":"2023-10-06T01:39:49.350613+0000","last_became_active":"2023-10-06T01:39:45.336383+0000","last_became_peered":"2023-10-06T01:39:45.336383+0000","last_unstale":"2023-10-06T01:39:49.350613+0000","last_undegraded":"2023-10-06T01:39:49.350613+0000","last_fullsized":"2023-10-06T01:39:49.350613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342395+0000","last_change":"2023-10-06T01:39:45.332271+0000","last_active":"2023-10-06T01:39:49.342395+0000","last_peered":"2023-10-06T01:39:49.342395+0000","last_clean":"2023-10-06T01:39:49.342395+0000","last_became_active":"2023-10-06T01:39:45.331844+0000","last_became_peered":"2023-10-06T01:39:45.331844+0000","last_unstale":"2023-10-06T01:39:49.342395+0000","last_undegraded":"2023-10-06T01:39:49.342395+0000","last_fullsized":"2023-10-06T01:39:49.342395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:01.791676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351382+0000","last_change":"2023-10-06T01:39:45.336079+0000","last_active":"2023-10-06T01:39:49.351382+0000","last_peered":"2023-10-06T01:39:49.351382+0000","last_clean":"2023-10-06T01:39:49.351382+0000","last_became_active":"2023-10-06T01:39:45.335828+0000","last_became_peered":"2023-10-06T01:39:45.335828+0000","last_unstale":"2023-10-06T01:39:49.351382+0000","last_undegraded":"2023-10-06T01:39:49.351382+0000","last_fullsized":"2023-10-06T01:39:49.351382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342168+0000","last_change":"2023-10-06T01:39:45.339626+0000","last_active":"2023-10-06T01:39:49.342168+0000","last_peered":"2023-10-06T01:39:49.342168+0000","last_clean":"2023-10-06T01:39:49.342168+0000","last_became_active":"2023-10-06T01:39:45.339370+0000","last_became_peered":"2023-10-06T01:39:45.339370+0000","last_unstale":"2023-10-06T01:39:49.342168+0000","last_undegraded":"2023-10-06T01:39:49.342168+0000","last_fullsized":"2023-10-06T01:39:49.342168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:12.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346086+0000","last_change":"2023-10-06T01:39:45.325821+0000","last_active":"2023-10-06T01:39:49.346086+0000","last_peered":"2023-10-06T01:39:49.346086+0000","last_clean":"2023-10-06T01:39:49.346086+0000","last_became_active":"2023-10-06T01:39:45.324033+0000","last_became_peered":"2023-10-06T01:39:45.324033+0000","last_unstale":"2023-10-06T01:39:49.346086+0000","last_undegraded":"2023-10-06T01:39:49.346086+0000","last_fullsized":"2023-10-06T01:39:49.346086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:13.558462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:45.330675+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:45.330406+0000","last_became_peered":"2023-10-06T01:39:45.330406+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350748+0000","last_change":"2023-10-06T01:39:45.342284+0000","last_active":"2023-10-06T01:39:49.350748+0000","last_peered":"2023-10-06T01:39:49.350748+0000","last_clean":"2023-10-06T01:39:49.350748+0000","last_became_active":"2023-10-06T01:39:45.342034+0000","last_became_peered":"2023-10-06T01:39:45.342034+0000","last_unstale":"2023-10-06T01:39:49.350748+0000","last_undegraded":"2023-10-06T01:39:49.350748+0000","last_fullsized":"2023-10-06T01:39:49.350748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347974+0000","last_change":"2023-10-06T01:39:45.334697+0000","last_active":"2023-10-06T01:39:49.347974+0000","last_peered":"2023-10-06T01:39:49.347974+0000","last_clean":"2023-10-06T01:39:49.347974+0000","last_became_active":"2023-10-06T01:39:45.334398+0000","last_became_peered":"2023-10-06T01:39:45.334398+0000","last_unstale":"2023-10-06T01:39:49.347974+0000","last_undegraded":"2023-10-06T01:39:49.347974+0000","last_fullsized":"2023-10-06T01:39:49.347974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338438+0000","last_change":"2023-10-06T01:39:45.336938+0000","last_active":"2023-10-06T01:39:49.338438+0000","last_peered":"2023-10-06T01:39:49.338438+0000","last_clean":"2023-10-06T01:39:49.338438+0000","last_became_active":"2023-10-06T01:39:45.336679+0000","last_became_peered":"2023-10-06T01:39:45.336679+0000","last_unstale":"2023-10-06T01:39:49.338438+0000","last_undegraded":"2023-10-06T01:39:49.338438+0000","last_fullsized":"2023-10-06T01:39:49.338438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350674+0000","last_change":"2023-10-06T01:39:45.338035+0000","last_active":"2023-10-06T01:39:49.350674+0000","last_peered":"2023-10-06T01:39:49.350674+0000","last_clean":"2023-10-06T01:39:49.350674+0000","last_became_active":"2023-10-06T01:39:45.337835+0000","last_became_peered":"2023-10-06T01:39:45.337835+0000","last_unstale":"2023-10-06T01:39:49.350674+0000","last_undegraded":"2023-10-06T01:39:49.350674+0000","last_fullsized":"2023-10-06T01:39:49.350674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344202+0000","last_change":"2023-10-06T01:39:45.328764+0000","last_active":"2023-10-06T01:39:49.344202+0000","last_peered":"2023-10-06T01:39:49.344202+0000","last_clean":"2023-10-06T01:39:49.344202+0000","last_became_active":"2023-10-06T01:39:45.328221+0000","last_became_peered":"2023-10-06T01:39:45.328221+0000","last_unstale":"2023-10-06T01:39:49.344202+0000","last_undegraded":"2023-10-06T01:39:49.344202+0000","last_fullsized":"2023-10-06T01:39:49.344202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344090+0000","last_change":"2023-10-06T01:39:45.337344+0000","last_active":"2023-10-06T01:39:49.344090+0000","last_peered":"2023-10-06T01:39:49.344090+0000","last_clean":"2023-10-06T01:39:49.344090+0000","last_became_active":"2023-10-06T01:39:45.336952+0000","last_became_peered":"2023-10-06T01:39:45.336952+0000","last_unstale":"2023-10-06T01:39:49.344090+0000","last_undegraded":"2023-10-06T01:39:49.344090+0000","last_fullsized":"2023-10-06T01:39:49.344090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350190+0000","last_change":"2023-10-06T01:39:45.334194+0000","last_active":"2023-10-06T01:39:49.350190+0000","last_peered":"2023-10-06T01:39:49.350190+0000","last_clean":"2023-10-06T01:39:49.350190+0000","last_became_active":"2023-10-06T01:39:45.333745+0000","last_became_peered":"2023-10-06T01:39:45.333745+0000","last_unstale":"2023-10-06T01:39:49.350190+0000","last_undegraded":"2023-10-06T01:39:49.350190+0000","last_fullsized":"2023-10-06T01:39:49.350190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360851+0000","last_change":"2023-10-06T01:39:49.359145+0000","last_active":"2023-10-06T01:39:49.360851+0000","last_peered":"2023-10-06T01:39:49.360851+0000","last_clean":"2023-10-06T01:39:49.360851+0000","last_became_active":"2023-10-06T01:39:49.358785+0000","last_became_peered":"2023-10-06T01:39:49.358785+0000","last_unstale":"2023-10-06T01:39:49.360851+0000","last_undegraded":"2023-10-06T01:39:49.360851+0000","last_fullsized":"2023-10-06T01:39:49.360851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:22.255025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340582+0000","last_change":"2023-10-06T01:39:44.325257+0000","last_active":"2023-10-06T01:39:49.340582+0000","last_peered":"2023-10-06T01:39:49.340582+0000","last_clean":"2023-10-06T01:39:49.340582+0000","last_became_active":"2023-10-06T01:39:44.324956+0000","last_became_peered":"2023-10-06T01:39:44.324956+0000","last_unstale":"2023-10-06T01:39:49.340582+0000","last_undegraded":"2023-10-06T01:39:49.340582+0000","last_fullsized":"2023-10-06T01:39:49.340582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338359+0000","last_change":"2023-10-06T01:39:45.331219+0000","last_active":"2023-10-06T01:39:49.338359+0000","last_peered":"2023-10-06T01:39:49.338359+0000","last_clean":"2023-10-06T01:39:49.338359+0000","last_became_active":"2023-10-06T01:39:45.330964+0000","last_became_peered":"2023-10-06T01:39:45.330964+0000","last_unstale":"2023-10-06T01:39:49.338359+0000","last_undegraded":"2023-10-06T01:39:49.338359+0000","last_fullsized":"2023-10-06T01:39:49.338359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359624+0000","last_change":"2023-10-06T01:39:49.356248+0000","last_active":"2023-10-06T01:39:49.359624+0000","last_peered":"2023-10-06T01:39:49.359624+0000","last_clean":"2023-10-06T01:39:49.359624+0000","last_became_active":"2023-10-06T01:39:49.355851+0000","last_became_peered":"2023-10-06T01:39:49.355851+0000","last_unstale":"2023-10-06T01:39:49.359624+0000","last_undegraded":"2023-10-06T01:39:49.359624+0000","last_fullsized":"2023-10-06T01:39:49.359624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341956+0000","last_change":"2023-10-06T01:39:44.327847+0000","last_active":"2023-10-06T01:39:49.341956+0000","last_peered":"2023-10-06T01:39:49.341956+0000","last_clean":"2023-10-06T01:39:49.341956+0000","last_became_active":"2023-10-06T01:39:44.327533+0000","last_became_peered":"2023-10-06T01:39:44.327533+0000","last_unstale":"2023-10-06T01:39:49.341956+0000","last_undegraded":"2023-10-06T01:39:49.341956+0000","last_fullsized":"2023-10-06T01:39:49.341956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351266+0000","last_change":"2023-10-06T01:39:45.326239+0000","last_active":"2023-10-06T01:39:49.351266+0000","last_peered":"2023-10-06T01:39:49.351266+0000","last_clean":"2023-10-06T01:39:49.351266+0000","last_became_active":"2023-10-06T01:39:45.325981+0000","last_became_peered":"2023-10-06T01:39:45.325981+0000","last_unstale":"2023-10-06T01:39:49.351266+0000","last_undegraded":"2023-10-06T01:39:49.351266+0000","last_fullsized":"2023-10-06T01:39:49.351266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361578+0000","last_change":"2023-10-06T01:39:49.358502+0000","last_active":"2023-10-06T01:39:49.361578+0000","last_peered":"2023-10-06T01:39:49.361578+0000","last_clean":"2023-10-06T01:39:49.361578+0000","last_became_active":"2023-10-06T01:39:49.358114+0000","last_became_peered":"2023-10-06T01:39:49.358114+0000","last_unstale":"2023-10-06T01:39:49.361578+0000","last_undegraded":"2023-10-06T01:39:49.361578+0000","last_fullsized":"2023-10-06T01:39:49.361578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338202+0000","last_change":"2023-10-06T01:39:44.325114+0000","last_active":"2023-10-06T01:39:49.338202+0000","last_peered":"2023-10-06T01:39:49.338202+0000","last_clean":"2023-10-06T01:39:49.338202+0000","last_became_active":"2023-10-06T01:39:44.324802+0000","last_became_peered":"2023-10-06T01:39:44.324802+0000","last_unstale":"2023-10-06T01:39:49.338202+0000","last_undegraded":"2023-10-06T01:39:49.338202+0000","last_fullsized":"2023-10-06T01:39:49.338202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:25.402016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339917+0000","last_change":"2023-10-06T01:39:45.337997+0000","last_active":"2023-10-06T01:39:49.339917+0000","last_peered":"2023-10-06T01:39:49.339917+0000","last_clean":"2023-10-06T01:39:49.339917+0000","last_became_active":"2023-10-06T01:39:45.337770+0000","last_became_peered":"2023-10-06T01:39:45.337770+0000","last_unstale":"2023-10-06T01:39:49.339917+0000","last_undegraded":"2023-10-06T01:39:49.339917+0000","last_fullsized":"2023-10-06T01:39:49.339917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359080+0000","last_change":"2023-10-06T01:39:49.355683+0000","last_active":"2023-10-06T01:39:49.359080+0000","last_peered":"2023-10-06T01:39:49.359080+0000","last_clean":"2023-10-06T01:39:49.359080+0000","last_became_active":"2023-10-06T01:39:49.355122+0000","last_became_peered":"2023-10-06T01:39:49.355122+0000","last_unstale":"2023-10-06T01:39:49.359080+0000","last_undegraded":"2023-10-06T01:39:49.359080+0000","last_fullsized":"2023-10-06T01:39:49.359080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351467+0000","last_change":"2023-10-06T01:39:44.326264+0000","last_active":"2023-10-06T01:39:49.351467+0000","last_peered":"2023-10-06T01:39:49.351467+0000","last_clean":"2023-10-06T01:39:49.351467+0000","last_became_active":"2023-10-06T01:39:44.325998+0000","last_became_peered":"2023-10-06T01:39:44.325998+0000","last_unstale":"2023-10-06T01:39:49.351467+0000","last_undegraded":"2023-10-06T01:39:49.351467+0000","last_fullsized":"2023-10-06T01:39:49.351467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338117+0000","last_change":"2023-10-06T01:39:45.332320+0000","last_active":"2023-10-06T01:39:49.338117+0000","last_peered":"2023-10-06T01:39:49.338117+0000","last_clean":"2023-10-06T01:39:49.338117+0000","last_became_active":"2023-10-06T01:39:45.332049+0000","last_became_peered":"2023-10-06T01:39:45.332049+0000","last_unstale":"2023-10-06T01:39:49.338117+0000","last_undegraded":"2023-10-06T01:39:49.338117+0000","last_fullsized":"2023-10-06T01:39:49.338117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363738+0000","last_change":"2023-10-06T01:39:49.360295+0000","last_active":"2023-10-06T01:39:49.363738+0000","last_peered":"2023-10-06T01:39:49.363738+0000","last_clean":"2023-10-06T01:39:49.363738+0000","last_became_active":"2023-10-06T01:39:49.359873+0000","last_became_peered":"2023-10-06T01:39:49.359873+0000","last_unstale":"2023-10-06T01:39:49.363738+0000","last_undegraded":"2023-10-06T01:39:49.363738+0000","last_fullsized":"2023-10-06T01:39:49.363738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356499+0000","last_change":"2023-10-06T01:39:44.328151+0000","last_active":"2023-10-06T01:39:49.356499+0000","last_peered":"2023-10-06T01:39:49.356499+0000","last_clean":"2023-10-06T01:39:49.356499+0000","last_became_active":"2023-10-06T01:39:44.327876+0000","last_became_peered":"2023-10-06T01:39:44.327876+0000","last_unstale":"2023-10-06T01:39:49.356499+0000","last_undegraded":"2023-10-06T01:39:49.356499+0000","last_fullsized":"2023-10-06T01:39:49.356499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337479+0000","last_change":"2023-10-06T01:39:45.330616+0000","last_active":"2023-10-06T01:39:49.337479+0000","last_peered":"2023-10-06T01:39:49.337479+0000","last_clean":"2023-10-06T01:39:49.337479+0000","last_became_active":"2023-10-06T01:39:45.330340+0000","last_became_peered":"2023-10-06T01:39:45.330340+0000","last_unstale":"2023-10-06T01:39:49.337479+0000","last_undegraded":"2023-10-06T01:39:49.337479+0000","last_fullsized":"2023-10-06T01:39:49.337479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368948+0000","last_change":"2023-10-06T01:39:49.365881+0000","last_active":"2023-10-06T01:39:49.368948+0000","last_peered":"2023-10-06T01:39:49.368948+0000","last_clean":"2023-10-06T01:39:49.368948+0000","last_became_active":"2023-10-06T01:39:49.365499+0000","last_became_peered":"2023-10-06T01:39:49.365499+0000","last_unstale":"2023-10-06T01:39:49.368948+0000","last_undegraded":"2023-10-06T01:39:49.368948+0000","last_fullsized":"2023-10-06T01:39:49.368948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:06.882659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351163+0000","last_change":"2023-10-06T01:39:44.326937+0000","last_active":"2023-10-06T01:39:49.351163+0000","last_peered":"2023-10-06T01:39:49.351163+0000","last_clean":"2023-10-06T01:39:49.351163+0000","last_became_active":"2023-10-06T01:39:44.326701+0000","last_became_peered":"2023-10-06T01:39:44.326701+0000","last_unstale":"2023-10-06T01:39:49.351163+0000","last_undegraded":"2023-10-06T01:39:49.351163+0000","last_fullsized":"2023-10-06T01:39:49.351163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:23.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342612+0000","last_change":"2023-10-06T01:39:45.331176+0000","last_active":"2023-10-06T01:39:49.342612+0000","last_peered":"2023-10-06T01:39:49.342612+0000","last_clean":"2023-10-06T01:39:49.342612+0000","last_became_active":"2023-10-06T01:39:45.330927+0000","last_became_peered":"2023-10-06T01:39:45.330927+0000","last_unstale":"2023-10-06T01:39:49.342612+0000","last_undegraded":"2023-10-06T01:39:49.342612+0000","last_fullsized":"2023-10-06T01:39:49.342612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357166+0000","last_change":"2023-10-06T01:39:49.355332+0000","last_active":"2023-10-06T01:39:49.357166+0000","last_peered":"2023-10-06T01:39:49.357166+0000","last_clean":"2023-10-06T01:39:49.357166+0000","last_became_active":"2023-10-06T01:39:49.354961+0000","last_became_peered":"2023-10-06T01:39:49.354961+0000","last_unstale":"2023-10-06T01:39:49.357166+0000","last_undegraded":"2023-10-06T01:39:49.357166+0000","last_fullsized":"2023-10-06T01:39:49.357166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T01:39:55.162081+0000","last_change":"2023-10-06T01:39:44.325901+0000","last_active":"2023-10-06T01:39:55.162081+0000","last_peered":"2023-10-06T01:39:55.162081+0000","last_clean":"2023-10-06T01:39:55.162081+0000","last_became_active":"2023-10-06T01:39:44.325631+0000","last_became_peered":"2023-10-06T01:39:44.325631+0000","last_unstale":"2023-10-06T01:39:55.162081+0000","last_undegraded":"2023-10-06T01:39:55.162081+0000","last_fullsized":"2023-10-06T01:39:55.162081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340321+0000","last_change":"2023-10-06T01:39:45.328644+0000","last_active":"2023-10-06T01:39:49.340321+0000","last_peered":"2023-10-06T01:39:49.340321+0000","last_clean":"2023-10-06T01:39:49.340321+0000","last_became_active":"2023-10-06T01:39:45.328367+0000","last_became_peered":"2023-10-06T01:39:45.328367+0000","last_unstale":"2023-10-06T01:39:49.340321+0000","last_undegraded":"2023-10-06T01:39:49.340321+0000","last_fullsized":"2023-10-06T01:39:49.340321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367347+0000","last_change":"2023-10-06T01:39:49.364574+0000","last_active":"2023-10-06T01:39:49.367347+0000","last_peered":"2023-10-06T01:39:49.367347+0000","last_clean":"2023-10-06T01:39:49.367347+0000","last_became_active":"2023-10-06T01:39:49.360749+0000","last_became_peered":"2023-10-06T01:39:49.360749+0000","last_unstale":"2023-10-06T01:39:49.367347+0000","last_undegraded":"2023-10-06T01:39:49.367347+0000","last_fullsized":"2023-10-06T01:39:49.367347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350570+0000","last_change":"2023-10-06T01:39:44.332362+0000","last_active":"2023-10-06T01:39:49.350570+0000","last_peered":"2023-10-06T01:39:49.350570+0000","last_clean":"2023-10-06T01:39:49.350570+0000","last_became_active":"2023-10-06T01:39:44.332082+0000","last_became_peered":"2023-10-06T01:39:44.332082+0000","last_unstale":"2023-10-06T01:39:49.350570+0000","last_undegraded":"2023-10-06T01:39:49.350570+0000","last_fullsized":"2023-10-06T01:39:49.350570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338444+0000","last_change":"2023-10-06T01:39:45.333142+0000","last_active":"2023-10-06T01:39:49.338444+0000","last_peered":"2023-10-06T01:39:49.338444+0000","last_clean":"2023-10-06T01:39:49.338444+0000","last_became_active":"2023-10-06T01:39:45.332880+0000","last_became_peered":"2023-10-06T01:39:45.332880+0000","last_unstale":"2023-10-06T01:39:49.338444+0000","last_undegraded":"2023-10-06T01:39:49.338444+0000","last_fullsized":"2023-10-06T01:39:49.338444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359341+0000","last_change":"2023-10-06T01:39:49.357412+0000","last_active":"2023-10-06T01:39:49.359341+0000","last_peered":"2023-10-06T01:39:49.359341+0000","last_clean":"2023-10-06T01:39:49.359341+0000","last_became_active":"2023-10-06T01:39:49.357031+0000","last_became_peered":"2023-10-06T01:39:49.357031+0000","last_unstale":"2023-10-06T01:39:49.359341+0000","last_undegraded":"2023-10-06T01:39:49.359341+0000","last_fullsized":"2023-10-06T01:39:49.359341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.348119+0000","last_change":"2023-10-06T01:39:44.325262+0000","last_active":"2023-10-06T01:39:49.348119+0000","last_peered":"2023-10-06T01:39:49.348119+0000","last_clean":"2023-10-06T01:39:49.348119+0000","last_became_active":"2023-10-06T01:39:44.325017+0000","last_became_peered":"2023-10-06T01:39:44.325017+0000","last_unstale":"2023-10-06T01:39:49.348119+0000","last_undegraded":"2023-10-06T01:39:49.348119+0000","last_fullsized":"2023-10-06T01:39:49.348119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350108+0000","last_change":"2023-10-06T01:39:45.334022+0000","last_active":"2023-10-06T01:39:49.350108+0000","last_peered":"2023-10-06T01:39:49.350108+0000","last_clean":"2023-10-06T01:39:49.350108+0000","last_became_active":"2023-10-06T01:39:45.333462+0000","last_became_peered":"2023-10-06T01:39:45.333462+0000","last_unstale":"2023-10-06T01:39:49.350108+0000","last_undegraded":"2023-10-06T01:39:49.350108+0000","last_fullsized":"2023-10-06T01:39:49.350108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.358648+0000","last_change":"2023-10-06T01:39:49.357484+0000","last_active":"2023-10-06T01:39:49.358648+0000","last_peered":"2023-10-06T01:39:49.358648+0000","last_clean":"2023-10-06T01:39:49.358648+0000","last_became_active":"2023-10-06T01:39:49.356839+0000","last_became_peered":"2023-10-06T01:39:49.356839+0000","last_unstale":"2023-10-06T01:39:49.358648+0000","last_undegraded":"2023-10-06T01:39:49.358648+0000","last_fullsized":"2023-10-06T01:39:49.358648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:14.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350742+0000","last_change":"2023-10-06T01:39:44.333226+0000","last_active":"2023-10-06T01:39:49.350742+0000","last_peered":"2023-10-06T01:39:49.350742+0000","last_clean":"2023-10-06T01:39:49.350742+0000","last_became_active":"2023-10-06T01:39:44.332920+0000","last_became_peered":"2023-10-06T01:39:44.332920+0000","last_unstale":"2023-10-06T01:39:49.350742+0000","last_undegraded":"2023-10-06T01:39:49.350742+0000","last_fullsized":"2023-10-06T01:39:49.350742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.344914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351529+0000","last_change":"2023-10-06T01:39:45.333240+0000","last_active":"2023-10-06T01:39:49.351529+0000","last_peered":"2023-10-06T01:39:49.351529+0000","last_clean":"2023-10-06T01:39:49.351529+0000","last_became_active":"2023-10-06T01:39:45.332985+0000","last_became_peered":"2023-10-06T01:39:45.332985+0000","last_unstale":"2023-10-06T01:39:49.351529+0000","last_undegraded":"2023-10-06T01:39:49.351529+0000","last_fullsized":"2023-10-06T01:39:49.351529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.364965+0000","last_change":"2023-10-06T01:39:49.360274+0000","last_active":"2023-10-06T01:39:49.364965+0000","last_peered":"2023-10-06T01:39:49.364965+0000","last_clean":"2023-10-06T01:39:49.364965+0000","last_became_active":"2023-10-06T01:39:49.359287+0000","last_became_peered":"2023-10-06T01:39:49.359287+0000","last_unstale":"2023-10-06T01:39:49.364965+0000","last_undegraded":"2023-10-06T01:39:49.364965+0000","last_fullsized":"2023-10-06T01:39:49.364965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:39.297464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343143+0000","last_change":"2023-10-06T01:39:44.322253+0000","last_active":"2023-10-06T01:39:49.343143+0000","last_peered":"2023-10-06T01:39:49.343143+0000","last_clean":"2023-10-06T01:39:49.343143+0000","last_became_active":"2023-10-06T01:39:44.321953+0000","last_became_peered":"2023-10-06T01:39:44.321953+0000","last_unstale":"2023-10-06T01:39:49.343143+0000","last_undegraded":"2023-10-06T01:39:49.343143+0000","last_fullsized":"2023-10-06T01:39:49.343143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338146+0000","last_change":"2023-10-06T01:39:45.343656+0000","last_active":"2023-10-06T01:39:49.338146+0000","last_peered":"2023-10-06T01:39:49.338146+0000","last_clean":"2023-10-06T01:39:49.338146+0000","last_became_active":"2023-10-06T01:39:45.342592+0000","last_became_peered":"2023-10-06T01:39:45.342592+0000","last_unstale":"2023-10-06T01:39:49.338146+0000","last_undegraded":"2023-10-06T01:39:49.338146+0000","last_fullsized":"2023-10-06T01:39:49.338146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365064+0000","last_change":"2023-10-06T01:39:49.363708+0000","last_active":"2023-10-06T01:39:49.365064+0000","last_peered":"2023-10-06T01:39:49.365064+0000","last_clean":"2023-10-06T01:39:49.365064+0000","last_became_active":"2023-10-06T01:39:49.363055+0000","last_became_peered":"2023-10-06T01:39:49.363055+0000","last_unstale":"2023-10-06T01:39:49.365064+0000","last_undegraded":"2023-10-06T01:39:49.365064+0000","last_fullsized":"2023-10-06T01:39:49.365064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352442+0000","last_change":"2023-10-06T01:39:44.333358+0000","last_active":"2023-10-06T01:39:49.352442+0000","last_peered":"2023-10-06T01:39:49.352442+0000","last_clean":"2023-10-06T01:39:49.352442+0000","last_became_active":"2023-10-06T01:39:44.332979+0000","last_became_peered":"2023-10-06T01:39:44.332979+0000","last_unstale":"2023-10-06T01:39:49.352442+0000","last_undegraded":"2023-10-06T01:39:49.352442+0000","last_fullsized":"2023-10-06T01:39:49.352442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349928+0000","last_change":"2023-10-06T01:39:45.330806+0000","last_active":"2023-10-06T01:39:49.349928+0000","last_peered":"2023-10-06T01:39:49.349928+0000","last_clean":"2023-10-06T01:39:49.349928+0000","last_became_active":"2023-10-06T01:39:45.330280+0000","last_became_peered":"2023-10-06T01:39:45.330280+0000","last_unstale":"2023-10-06T01:39:49.349928+0000","last_undegraded":"2023-10-06T01:39:49.349928+0000","last_fullsized":"2023-10-06T01:39:49.349928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.355843+0000","last_change":"2023-10-06T01:39:49.352785+0000","last_active":"2023-10-06T01:39:49.355843+0000","last_peered":"2023-10-06T01:39:49.355843+0000","last_clean":"2023-10-06T01:39:49.355843+0000","last_became_active":"2023-10-06T01:39:49.352396+0000","last_became_peered":"2023-10-06T01:39:49.352396+0000","last_unstale":"2023-10-06T01:39:49.355843+0000","last_undegraded":"2023-10-06T01:39:49.355843+0000","last_fullsized":"2023-10-06T01:39:49.355843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:45.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352263+0000","last_change":"2023-10-06T01:39:44.332277+0000","last_active":"2023-10-06T01:39:49.352263+0000","last_peered":"2023-10-06T01:39:49.352263+0000","last_clean":"2023-10-06T01:39:49.352263+0000","last_became_active":"2023-10-06T01:39:44.331987+0000","last_became_peered":"2023-10-06T01:39:44.331987+0000","last_unstale":"2023-10-06T01:39:49.352263+0000","last_undegraded":"2023-10-06T01:39:49.352263+0000","last_fullsized":"2023-10-06T01:39:49.352263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:36.012185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342030+0000","last_change":"2023-10-06T01:39:45.326893+0000","last_active":"2023-10-06T01:39:49.342030+0000","last_peered":"2023-10-06T01:39:49.342030+0000","last_clean":"2023-10-06T01:39:49.342030+0000","last_became_active":"2023-10-06T01:39:45.326610+0000","last_became_peered":"2023-10-06T01:39:45.326610+0000","last_unstale":"2023-10-06T01:39:49.342030+0000","last_undegraded":"2023-10-06T01:39:49.342030+0000","last_fullsized":"2023-10-06T01:39:49.342030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360816+0000","last_change":"2023-10-06T01:39:49.359772+0000","last_active":"2023-10-06T01:39:49.360816+0000","last_peered":"2023-10-06T01:39:49.360816+0000","last_clean":"2023-10-06T01:39:49.360816+0000","last_became_active":"2023-10-06T01:39:49.359414+0000","last_became_peered":"2023-10-06T01:39:49.359414+0000","last_unstale":"2023-10-06T01:39:49.360816+0000","last_undegraded":"2023-10-06T01:39:49.360816+0000","last_fullsized":"2023-10-06T01:39:49.360816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351061+0000","last_change":"2023-10-06T01:39:44.330033+0000","last_active":"2023-10-06T01:39:49.351061+0000","last_peered":"2023-10-06T01:39:49.351061+0000","last_clean":"2023-10-06T01:39:49.351061+0000","last_became_active":"2023-10-06T01:39:44.329732+0000","last_became_peered":"2023-10-06T01:39:44.329732+0000","last_unstale":"2023-10-06T01:39:49.351061+0000","last_undegraded":"2023-10-06T01:39:49.351061+0000","last_fullsized":"2023-10-06T01:39:49.351061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339868+0000","last_change":"2023-10-06T01:39:45.334185+0000","last_active":"2023-10-06T01:39:49.339868+0000","last_peered":"2023-10-06T01:39:49.339868+0000","last_clean":"2023-10-06T01:39:49.339868+0000","last_became_active":"2023-10-06T01:39:45.333053+0000","last_became_peered":"2023-10-06T01:39:45.333053+0000","last_unstale":"2023-10-06T01:39:49.339868+0000","last_undegraded":"2023-10-06T01:39:49.339868+0000","last_fullsized":"2023-10-06T01:39:49.339868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:58.429853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356192+0000","last_change":"2023-10-06T01:39:49.353291+0000","last_active":"2023-10-06T01:39:49.356192+0000","last_peered":"2023-10-06T01:39:49.356192+0000","last_clean":"2023-10-06T01:39:49.356192+0000","last_became_active":"2023-10-06T01:39:49.352912+0000","last_became_peered":"2023-10-06T01:39:49.352912+0000","last_unstale":"2023-10-06T01:39:49.356192+0000","last_undegraded":"2023-10-06T01:39:49.356192+0000","last_fullsized":"2023-10-06T01:39:49.356192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.229209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.325148+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.324687+0000","last_became_peered":"2023-10-06T01:39:44.324687+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347753+0000","last_change":"2023-10-06T01:39:45.329116+0000","last_active":"2023-10-06T01:39:49.347753+0000","last_peered":"2023-10-06T01:39:49.347753+0000","last_clean":"2023-10-06T01:39:49.347753+0000","last_became_active":"2023-10-06T01:39:45.328839+0000","last_became_peered":"2023-10-06T01:39:45.328839+0000","last_unstale":"2023-10-06T01:39:49.347753+0000","last_undegraded":"2023-10-06T01:39:49.347753+0000","last_fullsized":"2023-10-06T01:39:49.347753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356181+0000","last_change":"2023-10-06T01:39:49.353796+0000","last_active":"2023-10-06T01:39:49.356181+0000","last_peered":"2023-10-06T01:39:49.356181+0000","last_clean":"2023-10-06T01:39:49.356181+0000","last_became_active":"2023-10-06T01:39:49.353138+0000","last_became_peered":"2023-10-06T01:39:49.353138+0000","last_unstale":"2023-10-06T01:39:49.356181+0000","last_undegraded":"2023-10-06T01:39:49.356181+0000","last_fullsized":"2023-10-06T01:39:49.356181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337716+0000","last_change":"2023-10-06T01:39:44.329032+0000","last_active":"2023-10-06T01:39:49.337716+0000","last_peered":"2023-10-06T01:39:49.337716+0000","last_clean":"2023-10-06T01:39:49.337716+0000","last_became_active":"2023-10-06T01:39:44.328775+0000","last_became_peered":"2023-10-06T01:39:44.328775+0000","last_unstale":"2023-10-06T01:39:49.337716+0000","last_undegraded":"2023-10-06T01:39:49.337716+0000","last_fullsized":"2023-10-06T01:39:49.337716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331469+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.331152+0000","last_became_peered":"2023-10-06T01:39:45.331152+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356783+0000","last_change":"2023-10-06T01:39:49.351619+0000","last_active":"2023-10-06T01:39:49.356783+0000","last_peered":"2023-10-06T01:39:49.356783+0000","last_clean":"2023-10-06T01:39:49.356783+0000","last_became_active":"2023-10-06T01:39:49.351271+0000","last_became_peered":"2023-10-06T01:39:49.351271+0000","last_unstale":"2023-10-06T01:39:49.356783+0000","last_undegraded":"2023-10-06T01:39:49.356783+0000","last_fullsized":"2023-10-06T01:39:49.356783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:05.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339967+0000","last_change":"2023-10-06T01:39:44.321209+0000","last_active":"2023-10-06T01:39:49.339967+0000","last_peered":"2023-10-06T01:39:49.339967+0000","last_clean":"2023-10-06T01:39:49.339967+0000","last_became_active":"2023-10-06T01:39:44.320932+0000","last_became_peered":"2023-10-06T01:39:44.320932+0000","last_unstale":"2023-10-06T01:39:49.339967+0000","last_undegraded":"2023-10-06T01:39:49.339967+0000","last_fullsized":"2023-10-06T01:39:49.339967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339741+0000","last_change":"2023-10-06T01:39:45.339859+0000","last_active":"2023-10-06T01:39:49.339741+0000","last_peered":"2023-10-06T01:39:49.339741+0000","last_clean":"2023-10-06T01:39:49.339741+0000","last_became_active":"2023-10-06T01:39:45.339422+0000","last_became_peered":"2023-10-06T01:39:45.339422+0000","last_unstale":"2023-10-06T01:39:49.339741+0000","last_undegraded":"2023-10-06T01:39:49.339741+0000","last_fullsized":"2023-10-06T01:39:49.339741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:36.266399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367304+0000","last_change":"2023-10-06T01:39:49.361654+0000","last_active":"2023-10-06T01:39:49.367304+0000","last_peered":"2023-10-06T01:39:49.367304+0000","last_clean":"2023-10-06T01:39:49.367304+0000","last_became_active":"2023-10-06T01:39:49.361257+0000","last_became_peered":"2023-10-06T01:39:49.361257+0000","last_unstale":"2023-10-06T01:39:49.367304+0000","last_undegraded":"2023-10-06T01:39:49.367304+0000","last_fullsized":"2023-10-06T01:39:49.367304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:13:40.277782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341922+0000","last_change":"2023-10-06T01:39:44.326380+0000","last_active":"2023-10-06T01:39:49.341922+0000","last_peered":"2023-10-06T01:39:49.341922+0000","last_clean":"2023-10-06T01:39:49.341922+0000","last_became_active":"2023-10-06T01:39:44.326059+0000","last_became_peered":"2023-10-06T01:39:44.326059+0000","last_unstale":"2023-10-06T01:39:49.341922+0000","last_undegraded":"2023-10-06T01:39:49.341922+0000","last_fullsized":"2023-10-06T01:39:49.341922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342079+0000","last_change":"2023-10-06T01:39:45.329138+0000","last_active":"2023-10-06T01:39:49.342079+0000","last_peered":"2023-10-06T01:39:49.342079+0000","last_clean":"2023-10-06T01:39:49.342079+0000","last_became_active":"2023-10-06T01:39:45.328861+0000","last_became_peered":"2023-10-06T01:39:45.328861+0000","last_unstale":"2023-10-06T01:39:49.342079+0000","last_undegraded":"2023-10-06T01:39:49.342079+0000","last_fullsized":"2023-10-06T01:39:49.342079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365004+0000","last_change":"2023-10-06T01:39:49.360752+0000","last_active":"2023-10-06T01:39:49.365004+0000","last_peered":"2023-10-06T01:39:49.365004+0000","last_clean":"2023-10-06T01:39:49.365004+0000","last_became_active":"2023-10-06T01:39:49.360376+0000","last_became_peered":"2023-10-06T01:39:49.360376+0000","last_unstale":"2023-10-06T01:39:49.365004+0000","last_undegraded":"2023-10-06T01:39:49.365004+0000","last_fullsized":"2023-10-06T01:39:49.365004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.092106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338022+0000","last_change":"2023-10-06T01:39:44.325482+0000","last_active":"2023-10-06T01:39:49.338022+0000","last_peered":"2023-10-06T01:39:49.338022+0000","last_clean":"2023-10-06T01:39:49.338022+0000","last_became_active":"2023-10-06T01:39:44.325160+0000","last_became_peered":"2023-10-06T01:39:44.325160+0000","last_unstale":"2023-10-06T01:39:49.338022+0000","last_undegraded":"2023-10-06T01:39:49.338022+0000","last_fullsized":"2023-10-06T01:39:49.338022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344360+0000","last_change":"2023-10-06T01:39:45.342331+0000","last_active":"2023-10-06T01:39:49.344360+0000","last_peered":"2023-10-06T01:39:49.344360+0000","last_clean":"2023-10-06T01:39:49.344360+0000","last_became_active":"2023-10-06T01:39:45.342022+0000","last_became_peered":"2023-10-06T01:39:45.342022+0000","last_unstale":"2023-10-06T01:39:49.344360+0000","last_undegraded":"2023-10-06T01:39:49.344360+0000","last_fullsized":"2023-10-06T01:39:49.344360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357018+0000","last_change":"2023-10-06T01:39:49.355423+0000","last_active":"2023-10-06T01:39:49.357018+0000","last_peered":"2023-10-06T01:39:49.357018+0000","last_clean":"2023-10-06T01:39:49.357018+0000","last_became_active":"2023-10-06T01:39:49.355075+0000","last_became_peered":"2023-10-06T01:39:49.355075+0000","last_unstale":"2023-10-06T01:39:49.357018+0000","last_undegraded":"2023-10-06T01:39:49.357018+0000","last_fullsized":"2023-10-06T01:39:49.357018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:02.539610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338012+0000","last_change":"2023-10-06T01:39:44.328632+0000","last_active":"2023-10-06T01:39:49.338012+0000","last_peered":"2023-10-06T01:39:49.338012+0000","last_clean":"2023-10-06T01:39:49.338012+0000","last_became_active":"2023-10-06T01:39:44.326328+0000","last_became_peered":"2023-10-06T01:39:44.326328+0000","last_unstale":"2023-10-06T01:39:49.338012+0000","last_undegraded":"2023-10-06T01:39:49.338012+0000","last_fullsized":"2023-10-06T01:39:49.338012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339913+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:39:49.339913+0000","last_peered":"2023-10-06T01:39:49.339913+0000","last_clean":"2023-10-06T01:39:49.339913+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:39:49.339913+0000","last_undegraded":"2023-10-06T01:39:49.339913+0000","last_fullsized":"2023-10-06T01:39:49.339913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337712+0000","last_change":"2023-10-06T01:39:45.342039+0000","last_active":"2023-10-06T01:39:49.337712+0000","last_peered":"2023-10-06T01:39:49.337712+0000","last_clean":"2023-10-06T01:39:49.337712+0000","last_became_active":"2023-10-06T01:39:45.341746+0000","last_became_peered":"2023-10-06T01:39:45.341746+0000","last_unstale":"2023-10-06T01:39:49.337712+0000","last_undegraded":"2023-10-06T01:39:49.337712+0000","last_fullsized":"2023-10-06T01:39:49.337712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354087+0000","last_change":"2023-10-06T01:39:49.352264+0000","last_active":"2023-10-06T01:39:49.354087+0000","last_peered":"2023-10-06T01:39:49.354087+0000","last_clean":"2023-10-06T01:39:49.354087+0000","last_became_active":"2023-10-06T01:39:49.351974+0000","last_became_peered":"2023-10-06T01:39:49.351974+0000","last_unstale":"2023-10-06T01:39:49.354087+0000","last_undegraded":"2023-10-06T01:39:49.354087+0000","last_fullsized":"2023-10-06T01:39:49.354087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:35.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337764+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:39:49.337764+0000","last_peered":"2023-10-06T01:39:49.337764+0000","last_clean":"2023-10-06T01:39:49.337764+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:39:49.337764+0000","last_undegraded":"2023-10-06T01:39:49.337764+0000","last_fullsized":"2023-10-06T01:39:49.337764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.395905+0000","last_change":"2023-10-06T01:39:44.324833+0000","last_active":"2023-10-06T01:39:49.395905+0000","last_peered":"2023-10-06T01:39:49.395905+0000","last_clean":"2023-10-06T01:39:49.395905+0000","last_became_active":"2023-10-06T01:39:44.324554+0000","last_became_peered":"2023-10-06T01:39:44.324554+0000","last_unstale":"2023-10-06T01:39:49.395905+0000","last_undegraded":"2023-10-06T01:39:49.395905+0000","last_fullsized":"2023-10-06T01:39:49.395905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:53.581079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341933+0000","last_change":"2023-10-06T01:39:45.331099+0000","last_active":"2023-10-06T01:39:49.341933+0000","last_peered":"2023-10-06T01:39:49.341933+0000","last_clean":"2023-10-06T01:39:49.341933+0000","last_became_active":"2023-10-06T01:39:45.330844+0000","last_became_peered":"2023-10-06T01:39:45.330844+0000","last_unstale":"2023-10-06T01:39:49.341933+0000","last_undegraded":"2023-10-06T01:39:49.341933+0000","last_fullsized":"2023-10-06T01:39:49.341933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367282+0000","last_change":"2023-10-06T01:39:49.360839+0000","last_active":"2023-10-06T01:39:49.367282+0000","last_peered":"2023-10-06T01:39:49.367282+0000","last_clean":"2023-10-06T01:39:49.367282+0000","last_became_active":"2023-10-06T01:39:49.360447+0000","last_became_peered":"2023-10-06T01:39:49.360447+0000","last_unstale":"2023-10-06T01:39:49.367282+0000","last_undegraded":"2023-10-06T01:39:49.367282+0000","last_fullsized":"2023-10-06T01:39:49.367282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342223+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:39:49.342223+0000","last_peered":"2023-10-06T01:39:49.342223+0000","last_clean":"2023-10-06T01:39:49.342223+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:39:49.342223+0000","last_undegraded":"2023-10-06T01:39:49.342223+0000","last_fullsized":"2023-10-06T01:39:49.342223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350783+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:39:49.350783+0000","last_peered":"2023-10-06T01:39:49.350783+0000","last_clean":"2023-10-06T01:39:49.350783+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:39:49.350783+0000","last_undegraded":"2023-10-06T01:39:49.350783+0000","last_fullsized":"2023-10-06T01:39:49.350783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346294+0000","last_change":"2023-10-06T01:39:44.332942+0000","last_active":"2023-10-06T01:39:49.346294+0000","last_peered":"2023-10-06T01:39:49.346294+0000","last_clean":"2023-10-06T01:39:49.346294+0000","last_became_active":"2023-10-06T01:39:44.332767+0000","last_became_peered":"2023-10-06T01:39:44.332767+0000","last_unstale":"2023-10-06T01:39:49.346294+0000","last_undegraded":"2023-10-06T01:39:49.346294+0000","last_fullsized":"2023-10-06T01:39:49.346294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351451+0000","last_change":"2023-10-06T01:39:45.331974+0000","last_active":"2023-10-06T01:39:49.351451+0000","last_peered":"2023-10-06T01:39:49.351451+0000","last_clean":"2023-10-06T01:39:49.351451+0000","last_became_active":"2023-10-06T01:39:45.331706+0000","last_became_peered":"2023-10-06T01:39:45.331706+0000","last_unstale":"2023-10-06T01:39:49.351451+0000","last_undegraded":"2023-10-06T01:39:49.351451+0000","last_fullsized":"2023-10-06T01:39:49.351451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357328+0000","last_change":"2023-10-06T01:39:49.356514+0000","last_active":"2023-10-06T01:39:49.357328+0000","last_peered":"2023-10-06T01:39:49.357328+0000","last_clean":"2023-10-06T01:39:49.357328+0000","last_became_active":"2023-10-06T01:39:49.356017+0000","last_became_peered":"2023-10-06T01:39:49.356017+0000","last_unstale":"2023-10-06T01:39:49.357328+0000","last_undegraded":"2023-10-06T01:39:49.357328+0000","last_fullsized":"2023-10-06T01:39:49.357328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:28.616739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350977+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:39:49.350977+0000","last_peered":"2023-10-06T01:39:49.350977+0000","last_clean":"2023-10-06T01:39:49.350977+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:39:49.350977+0000","last_undegraded":"2023-10-06T01:39:49.350977+0000","last_fullsized":"2023-10-06T01:39:49.350977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345374+0000","last_change":"2023-10-06T01:39:44.324538+0000","last_active":"2023-10-06T01:39:49.345374+0000","last_peered":"2023-10-06T01:39:49.345374+0000","last_clean":"2023-10-06T01:39:49.345374+0000","last_became_active":"2023-10-06T01:39:44.324239+0000","last_became_peered":"2023-10-06T01:39:44.324239+0000","last_unstale":"2023-10-06T01:39:49.345374+0000","last_undegraded":"2023-10-06T01:39:49.345374+0000","last_fullsized":"2023-10-06T01:39:49.345374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344282+0000","last_change":"2023-10-06T01:39:45.339432+0000","last_active":"2023-10-06T01:39:49.344282+0000","last_peered":"2023-10-06T01:39:49.344282+0000","last_clean":"2023-10-06T01:39:49.344282+0000","last_became_active":"2023-10-06T01:39:45.339191+0000","last_became_peered":"2023-10-06T01:39:45.339191+0000","last_unstale":"2023-10-06T01:39:49.344282+0000","last_undegraded":"2023-10-06T01:39:49.344282+0000","last_fullsized":"2023-10-06T01:39:49.344282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363746+0000","last_change":"2023-10-06T01:39:49.361536+0000","last_active":"2023-10-06T01:39:49.363746+0000","last_peered":"2023-10-06T01:39:49.363746+0000","last_clean":"2023-10-06T01:39:49.363746+0000","last_became_active":"2023-10-06T01:39:49.361162+0000","last_became_peered":"2023-10-06T01:39:49.361162+0000","last_unstale":"2023-10-06T01:39:49.363746+0000","last_undegraded":"2023-10-06T01:39:49.363746+0000","last_fullsized":"2023-10-06T01:39:49.363746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350576+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:39:49.350576+0000","last_peered":"2023-10-06T01:39:49.350576+0000","last_clean":"2023-10-06T01:39:49.350576+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:39:49.350576+0000","last_undegraded":"2023-10-06T01:39:49.350576+0000","last_fullsized":"2023-10-06T01:39:49.350576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337530+0000","last_change":"2023-10-06T01:39:44.328264+0000","last_active":"2023-10-06T01:39:49.337530+0000","last_peered":"2023-10-06T01:39:49.337530+0000","last_clean":"2023-10-06T01:39:49.337530+0000","last_became_active":"2023-10-06T01:39:44.327969+0000","last_became_peered":"2023-10-06T01:39:44.327969+0000","last_unstale":"2023-10-06T01:39:49.337530+0000","last_undegraded":"2023-10-06T01:39:49.337530+0000","last_fullsized":"2023-10-06T01:39:49.337530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340324+0000","last_change":"2023-10-06T01:39:45.333753+0000","last_active":"2023-10-06T01:39:49.340324+0000","last_peered":"2023-10-06T01:39:49.340324+0000","last_clean":"2023-10-06T01:39:49.340324+0000","last_became_active":"2023-10-06T01:39:45.333416+0000","last_became_peered":"2023-10-06T01:39:45.333416+0000","last_unstale":"2023-10-06T01:39:49.340324+0000","last_undegraded":"2023-10-06T01:39:49.340324+0000","last_fullsized":"2023-10-06T01:39:49.340324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361707+0000","last_change":"2023-10-06T01:39:49.360037+0000","last_active":"2023-10-06T01:39:49.361707+0000","last_peered":"2023-10-06T01:39:49.361707+0000","last_clean":"2023-10-06T01:39:49.361707+0000","last_became_active":"2023-10-06T01:39:49.359697+0000","last_became_peered":"2023-10-06T01:39:49.359697+0000","last_unstale":"2023-10-06T01:39:49.361707+0000","last_undegraded":"2023-10-06T01:39:49.361707+0000","last_fullsized":"2023-10-06T01:39:49.361707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349757+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:39:49.349757+0000","last_peered":"2023-10-06T01:39:49.349757+0000","last_clean":"2023-10-06T01:39:49.349757+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:39:49.349757+0000","last_undegraded":"2023-10-06T01:39:49.349757+0000","last_fullsized":"2023-10-06T01:39:49.349757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:44.329991+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:44.329756+0000","last_became_peered":"2023-10-06T01:39:44.329756+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343619+0000","last_change":"2023-10-06T01:39:45.336684+0000","last_active":"2023-10-06T01:39:49.343619+0000","last_peered":"2023-10-06T01:39:49.343619+0000","last_clean":"2023-10-06T01:39:49.343619+0000","last_became_active":"2023-10-06T01:39:45.336427+0000","last_became_peered":"2023-10-06T01:39:45.336427+0000","last_unstale":"2023-10-06T01:39:49.343619+0000","last_undegraded":"2023-10-06T01:39:49.343619+0000","last_fullsized":"2023-10-06T01:39:49.343619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359866+0000","last_change":"2023-10-06T01:39:49.358138+0000","last_active":"2023-10-06T01:39:49.359866+0000","last_peered":"2023-10-06T01:39:49.359866+0000","last_clean":"2023-10-06T01:39:49.359866+0000","last_became_active":"2023-10-06T01:39:49.356144+0000","last_became_peered":"2023-10-06T01:39:49.356144+0000","last_unstale":"2023-10-06T01:39:49.359866+0000","last_undegraded":"2023-10-06T01:39:49.359866+0000","last_fullsized":"2023-10-06T01:39:49.359866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:02.622952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341234+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:39:49.341234+0000","last_peered":"2023-10-06T01:39:49.341234+0000","last_clean":"2023-10-06T01:39:49.341234+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:39:49.341234+0000","last_undegraded":"2023-10-06T01:39:49.341234+0000","last_fullsized":"2023-10-06T01:39:49.341234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343734+0000","last_change":"2023-10-06T01:39:44.327413+0000","last_active":"2023-10-06T01:39:49.343734+0000","last_peered":"2023-10-06T01:39:49.343734+0000","last_clean":"2023-10-06T01:39:49.343734+0000","last_became_active":"2023-10-06T01:39:44.327177+0000","last_became_peered":"2023-10-06T01:39:44.327177+0000","last_unstale":"2023-10-06T01:39:49.343734+0000","last_undegraded":"2023-10-06T01:39:49.343734+0000","last_fullsized":"2023-10-06T01:39:49.343734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:17.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347454+0000","last_change":"2023-10-06T01:39:45.334858+0000","last_active":"2023-10-06T01:39:49.347454+0000","last_peered":"2023-10-06T01:39:49.347454+0000","last_clean":"2023-10-06T01:39:49.347454+0000","last_became_active":"2023-10-06T01:39:45.334586+0000","last_became_peered":"2023-10-06T01:39:45.334586+0000","last_unstale":"2023-10-06T01:39:49.347454+0000","last_undegraded":"2023-10-06T01:39:49.347454+0000","last_fullsized":"2023-10-06T01:39:49.347454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:10.350398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.366854+0000","last_change":"2023-10-06T01:39:49.365711+0000","last_active":"2023-10-06T01:39:49.366854+0000","last_peered":"2023-10-06T01:39:49.366854+0000","last_clean":"2023-10-06T01:39:49.366854+0000","last_became_active":"2023-10-06T01:39:49.365357+0000","last_became_peered":"2023-10-06T01:39:49.365357+0000","last_unstale":"2023-10-06T01:39:49.366854+0000","last_undegraded":"2023-10-06T01:39:49.366854+0000","last_fullsized":"2023-10-06T01:39:49.366854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339829+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:39:49.339829+0000","last_peered":"2023-10-06T01:39:49.339829+0000","last_clean":"2023-10-06T01:39:49.339829+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:39:49.339829+0000","last_undegraded":"2023-10-06T01:39:49.339829+0000","last_fullsized":"2023-10-06T01:39:49.339829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350343+0000","last_change":"2023-10-06T01:39:44.327795+0000","last_active":"2023-10-06T01:39:49.350343+0000","last_peered":"2023-10-06T01:39:49.350343+0000","last_clean":"2023-10-06T01:39:49.350343+0000","last_became_active":"2023-10-06T01:39:44.326791+0000","last_became_peered":"2023-10-06T01:39:44.326791+0000","last_unstale":"2023-10-06T01:39:49.350343+0000","last_undegraded":"2023-10-06T01:39:49.350343+0000","last_fullsized":"2023-10-06T01:39:49.350343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342538+0000","last_change":"2023-10-06T01:39:45.341418+0000","last_active":"2023-10-06T01:39:49.342538+0000","last_peered":"2023-10-06T01:39:49.342538+0000","last_clean":"2023-10-06T01:39:49.342538+0000","last_became_active":"2023-10-06T01:39:45.341053+0000","last_became_peered":"2023-10-06T01:39:45.341053+0000","last_unstale":"2023-10-06T01:39:49.342538+0000","last_undegraded":"2023-10-06T01:39:49.342538+0000","last_fullsized":"2023-10-06T01:39:49.342538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356900+0000","last_change":"2023-10-06T01:39:49.355119+0000","last_active":"2023-10-06T01:39:49.356900+0000","last_peered":"2023-10-06T01:39:49.356900+0000","last_clean":"2023-10-06T01:39:49.356900+0000","last_became_active":"2023-10-06T01:39:49.354770+0000","last_became_peered":"2023-10-06T01:39:49.354770+0000","last_unstale":"2023-10-06T01:39:49.356900+0000","last_undegraded":"2023-10-06T01:39:49.356900+0000","last_fullsized":"2023-10-06T01:39:49.356900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:40.295194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347370+0000","last_change":"2023-10-06T01:39:44.333929+0000","last_active":"2023-10-06T01:39:49.347370+0000","last_peered":"2023-10-06T01:39:49.347370+0000","last_clean":"2023-10-06T01:39:49.347370+0000","last_became_active":"2023-10-06T01:39:44.333659+0000","last_became_peered":"2023-10-06T01:39:44.333659+0000","last_unstale":"2023-10-06T01:39:49.347370+0000","last_undegraded":"2023-10-06T01:39:49.347370+0000","last_fullsized":"2023-10-06T01:39:49.347370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346317+0000","last_change":"2023-10-06T01:39:45.331228+0000","last_active":"2023-10-06T01:39:49.346317+0000","last_peered":"2023-10-06T01:39:49.346317+0000","last_clean":"2023-10-06T01:39:49.346317+0000","last_became_active":"2023-10-06T01:39:45.330982+0000","last_became_peered":"2023-10-06T01:39:45.330982+0000","last_unstale":"2023-10-06T01:39:49.346317+0000","last_undegraded":"2023-10-06T01:39:49.346317+0000","last_fullsized":"2023-10-06T01:39:49.346317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.923135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363774+0000","last_change":"2023-10-06T01:39:49.362572+0000","last_active":"2023-10-06T01:39:49.363774+0000","last_peered":"2023-10-06T01:39:49.363774+0000","last_clean":"2023-10-06T01:39:49.363774+0000","last_became_active":"2023-10-06T01:39:49.362200+0000","last_became_peered":"2023-10-06T01:39:49.362200+0000","last_unstale":"2023-10-06T01:39:49.363774+0000","last_undegraded":"2023-10-06T01:39:49.363774+0000","last_fullsized":"2023-10-06T01:39:49.363774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349796+0000","last_change":"2023-10-06T01:39:44.332451+0000","last_active":"2023-10-06T01:39:49.349796+0000","last_peered":"2023-10-06T01:39:49.349796+0000","last_clean":"2023-10-06T01:39:49.349796+0000","last_became_active":"2023-10-06T01:39:44.332143+0000","last_became_peered":"2023-10-06T01:39:44.332143+0000","last_unstale":"2023-10-06T01:39:49.349796+0000","last_undegraded":"2023-10-06T01:39:49.349796+0000","last_fullsized":"2023-10-06T01:39:49.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331260+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.330980+0000","last_became_peered":"2023-10-06T01:39:45.330980+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368960+0000","last_change":"2023-10-06T01:39:49.361125+0000","last_active":"2023-10-06T01:39:49.368960+0000","last_peered":"2023-10-06T01:39:49.368960+0000","last_clean":"2023-10-06T01:39:49.368960+0000","last_became_active":"2023-10-06T01:39:49.354967+0000","last_became_peered":"2023-10-06T01:39:49.354967+0000","last_unstale":"2023-10-06T01:39:49.368960+0000","last_undegraded":"2023-10-06T01:39:49.368960+0000","last_fullsized":"2023-10-06T01:39:49.368960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:41.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.329353+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.329098+0000","last_became_peered":"2023-10-06T01:39:44.329098+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340491+0000","last_change":"2023-10-06T01:39:45.330780+0000","last_active":"2023-10-06T01:39:49.340491+0000","last_peered":"2023-10-06T01:39:49.340491+0000","last_clean":"2023-10-06T01:39:49.340491+0000","last_became_active":"2023-10-06T01:39:45.330484+0000","last_became_peered":"2023-10-06T01:39:45.330484+0000","last_unstale":"2023-10-06T01:39:49.340491+0000","last_undegraded":"2023-10-06T01:39:49.340491+0000","last_fullsized":"2023-10-06T01:39:49.340491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354659+0000","last_change":"2023-10-06T01:39:49.351706+0000","last_active":"2023-10-06T01:39:49.354659+0000","last_peered":"2023-10-06T01:39:49.354659+0000","last_clean":"2023-10-06T01:39:49.354659+0000","last_became_active":"2023-10-06T01:39:49.350630+0000","last_became_peered":"2023-10-06T01:39:49.350630+0000","last_unstale":"2023-10-06T01:39:49.354659+0000","last_undegraded":"2023-10-06T01:39:49.354659+0000","last_fullsized":"2023-10-06T01:39:49.354659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:57.336637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337644+0000","last_change":"2023-10-06T01:39:44.321387+0000","last_active":"2023-10-06T01:39:49.337644+0000","last_peered":"2023-10-06T01:39:49.337644+0000","last_clean":"2023-10-06T01:39:49.337644+0000","last_became_active":"2023-10-06T01:39:44.321101+0000","last_became_peered":"2023-10-06T01:39:44.321101+0000","last_unstale":"2023-10-06T01:39:49.337644+0000","last_undegraded":"2023-10-06T01:39:49.337644+0000","last_fullsized":"2023-10-06T01:39:49.337644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337678+0000","last_change":"2023-10-06T01:39:45.333689+0000","last_active":"2023-10-06T01:39:49.337678+0000","last_peered":"2023-10-06T01:39:49.337678+0000","last_clean":"2023-10-06T01:39:49.337678+0000","last_became_active":"2023-10-06T01:39:45.333444+0000","last_became_peered":"2023-10-06T01:39:45.333444+0000","last_unstale":"2023-10-06T01:39:49.337678+0000","last_undegraded":"2023-10-06T01:39:49.337678+0000","last_fullsized":"2023-10-06T01:39:49.337678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.362749+0000","last_change":"2023-10-06T01:39:49.360875+0000","last_active":"2023-10-06T01:39:49.362749+0000","last_peered":"2023-10-06T01:39:49.362749+0000","last_clean":"2023-10-06T01:39:49.362749+0000","last_became_active":"2023-10-06T01:39:49.360515+0000","last_became_peered":"2023-10-06T01:39:49.360515+0000","last_unstale":"2023-10-06T01:39:49.362749+0000","last_undegraded":"2023-10-06T01:39:49.362749+0000","last_fullsized":"2023-10-06T01:39:49.362749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351554+0000","last_change":"2023-10-06T01:39:44.326433+0000","last_active":"2023-10-06T01:39:49.351554+0000","last_peered":"2023-10-06T01:39:49.351554+0000","last_clean":"2023-10-06T01:39:49.351554+0000","last_became_active":"2023-10-06T01:39:44.326167+0000","last_became_peered":"2023-10-06T01:39:44.326167+0000","last_unstale":"2023-10-06T01:39:49.351554+0000","last_undegraded":"2023-10-06T01:39:49.351554+0000","last_fullsized":"2023-10-06T01:39:49.351554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342356+0000","last_change":"2023-10-06T01:39:45.330018+0000","last_active":"2023-10-06T01:39:49.342356+0000","last_peered":"2023-10-06T01:39:49.342356+0000","last_clean":"2023-10-06T01:39:49.342356+0000","last_became_active":"2023-10-06T01:39:45.329556+0000","last_became_peered":"2023-10-06T01:39:45.329556+0000","last_unstale":"2023-10-06T01:39:49.342356+0000","last_undegraded":"2023-10-06T01:39:49.342356+0000","last_fullsized":"2023-10-06T01:39:49.342356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343235+0000","last_change":"2023-10-06T01:39:45.333252+0000","last_active":"2023-10-06T01:39:49.343235+0000","last_peered":"2023-10-06T01:39:49.343235+0000","last_clean":"2023-10-06T01:39:49.343235+0000","last_became_active":"2023-10-06T01:39:45.332899+0000","last_became_peered":"2023-10-06T01:39:45.332899+0000","last_unstale":"2023-10-06T01:39:49.343235+0000","last_undegraded":"2023-10-06T01:39:49.343235+0000","last_fullsized":"2023-10-06T01:39:49.343235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:37.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337491+0000","last_change":"2023-10-06T01:39:45.342104+0000","last_active":"2023-10-06T01:39:49.337491+0000","last_peered":"2023-10-06T01:39:49.337491+0000","last_clean":"2023-10-06T01:39:49.337491+0000","last_became_active":"2023-10-06T01:39:45.341825+0000","last_became_peered":"2023-10-06T01:39:45.341825+0000","last_unstale":"2023-10-06T01:39:49.337491+0000","last_undegraded":"2023-10-06T01:39:49.337491+0000","last_fullsized":"2023-10-06T01:39:49.337491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340694+0000","last_change":"2023-10-06T01:39:45.329243+0000","last_active":"2023-10-06T01:39:49.340694+0000","last_peered":"2023-10-06T01:39:49.340694+0000","last_clean":"2023-10-06T01:39:49.340694+0000","last_became_active":"2023-10-06T01:39:45.328438+0000","last_became_peered":"2023-10-06T01:39:45.328438+0000","last_unstale":"2023-10-06T01:39:49.340694+0000","last_undegraded":"2023-10-06T01:39:49.340694+0000","last_fullsized":"2023-10-06T01:39:49.340694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337590+0000","last_change":"2023-10-06T01:39:45.330664+0000","last_active":"2023-10-06T01:39:49.337590+0000","last_peered":"2023-10-06T01:39:49.337590+0000","last_clean":"2023-10-06T01:39:49.337590+0000","last_became_active":"2023-10-06T01:39:45.330394+0000","last_became_peered":"2023-10-06T01:39:45.330394+0000","last_unstale":"2023-10-06T01:39:49.337590+0000","last_undegraded":"2023-10-06T01:39:49.337590+0000","last_fullsized":"2023-10-06T01:39:49.337590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343700+0000","last_change":"2023-10-06T01:39:45.338933+0000","last_active":"2023-10-06T01:39:49.343700+0000","last_peered":"2023-10-06T01:39:49.343700+0000","last_clean":"2023-10-06T01:39:49.343700+0000","last_became_active":"2023-10-06T01:39:45.338515+0000","last_became_peered":"2023-10-06T01:39:45.338515+0000","last_unstale":"2023-10-06T01:39:49.343700+0000","last_undegraded":"2023-10-06T01:39:49.343700+0000","last_fullsized":"2023-10-06T01:39:49.343700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":6791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":696558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:39:59.428 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T01:39:59.428 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T01:39:59.428 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T01:39:59.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T01:39:59.428 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T01:39:59.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.573+0000 7efc2e1b9700 1 Processor -- start 2023-10-06T01:39:59.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.573+0000 7efc2e1b9700 1 -- start start 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.574+0000 7efc2e1b9700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc280aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.574+0000 7efc2e1b9700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc280b07a0 0x7efc280b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.574+0000 7efc2e1b9700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc280b0ca0 con 0x7efc280b49e0 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.574+0000 7efc2e1b9700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280b0f80 con 0x7efc280b07a0 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.574+0000 7efc2e1b9700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b1260 con 0x7efc281595a0 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc280b07a0 0x7efc280b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc277fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc280aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc280b07a0 0x7efc280b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49742/0 (socket says 172.21.15.98:49742) 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 -- 172.21.15.98:0/2648188424 learned_addr learned my addr 172.21.15.98:0/2648188424 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 -- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 msgr2=0x7efc280aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc280aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 -- 172.21.15.98:0/2648188424 >> v1:172.21.15.184:6789/0 conn(0x7efc280b49e0 legacy=0x7efc280aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 -- 172.21.15.98:0/2648188424 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280b1540 con 0x7efc280b07a0 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc277fe700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc280aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc27fff700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc280b1c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc20009f20 tx=0x7efc20009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cab9a1f39791b14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc267fc700 1 -- 172.21.15.98:0/2648188424 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000e070 con 0x7efc280b07a0 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc267fc700 1 -- 172.21.15.98:0/2648188424 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2000b9d0 con 0x7efc280b07a0 2023-10-06T01:39:59.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.575+0000 7efc267fc700 1 -- 172.21.15.98:0/2648188424 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200092e0 con 0x7efc280b07a0 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 msgr2=0x7efc280b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc280b1c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc20009f20 tx=0x7efc20009840 comp rx=0 tx=0).stop 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2648188424 shutdown_connections 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc280b1c00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2648188424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc280aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2648188424 >> 172.21.15.98:0/2648188424 conn(0x7efc28099830 msgr2=0x7efc2806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2648188424 shutdown_connections 2023-10-06T01:39:59.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.576+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2648188424 wait complete. 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.577+0000 7efc2e1b9700 1 Processor -- start 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.577+0000 7efc2e1b9700 1 -- start start 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc280b49e0 0x7efc2816a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc2816c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2816a640 con 0x7efc280b49e0 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2816adf0 con 0x7efc280b07a0 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc2e1b9700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280ac7b0 con 0x7efc281595a0 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc277fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc277fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49758/0 (socket says 172.21.15.98:49758) 2023-10-06T01:39:59.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc277fe700 1 -- 172.21.15.98:0/2229128230 learned_addr learned my addr 172.21.15.98:0/2229128230 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc27fff700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc2816c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.578+0000 7efc26ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc280b49e0 0x7efc2816a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 -- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 msgr2=0x7efc2816c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc2816c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 -- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc280b49e0 msgr2=0x7efc2816a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc280b49e0 0x7efc2816a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 -- 172.21.15.98:0/2229128230 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180093c0 con 0x7efc280b07a0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc277fe700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc1800ec80 tx=0x7efc18009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18005a00 con 0x7efc280b07a0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18005b60 con 0x7efc280b07a0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc20009ae0 con 0x7efc280b07a0 2023-10-06T01:39:59.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800c850 con 0x7efc280b07a0 2023-10-06T01:39:59.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.579+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc280acd70 con 0x7efc280b07a0 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.580+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc28064ad0 con 0x7efc280b07a0 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.581+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc1800cbc0 con 0x7efc280b07a0 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.581+0000 7efc0f7fe700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 0x7efc140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.581+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7efc18084df0 con 0x7efc280b07a0 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.581+0000 7efc26ffd700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 0x7efc140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.581+0000 7efc26ffd700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 0x7efc140693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc280b4df0 tx=0x7efc1c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.587+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc18051b70 con 0x7efc280b07a0 2023-10-06T01:39:59.822 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.821+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efc2804bb80 con 0x7efc280b07a0 2023-10-06T01:39:59.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.825+0000 7efc0f7fe700 1 -- 172.21.15.98:0/2229128230 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163700 (secure 0 0 0) 0x7efc18055190 con 0x7efc280b07a0 2023-10-06T01:39:59.830 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:39:59.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 msgr2=0x7efc140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 0x7efc140693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc280b4df0 tx=0x7efc1c007ec0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 msgr2=0x7efc2816a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc1800ec80 tx=0x7efc18009120 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 shutdown_connections 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efc14066f30 0x7efc140693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc281595a0 0x7efc2816c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc280b07a0 0x7efc2816a100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 --2- 172.21.15.98:0/2229128230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc280b49e0 0x7efc2816a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 >> 172.21.15.98:0/2229128230 conn(0x7efc28099830 msgr2=0x7efc280b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 shutdown_connections 2023-10-06T01:39:59.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.827+0000 7efc2e1b9700 1 -- 172.21.15.98:0/2229128230 wait complete. 2023-10-06T01:39:59.840 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":120423},{"type":"v1","addr":"172.21.15.98:6833","nonce":120423}]},"active_addr":"172.21.15.98:6833/120423","active_change":"2023-10-06T01:39:31.777648+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":819040204}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":2099925390}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":1845616959}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":648989688}]}]} 2023-10-06T01:39:59.844 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T01:39:59.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T01:39:59.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:39:59.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.957+0000 7fe17adde700 1 Processor -- start 2023-10-06T01:39:59.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.957+0000 7fe17adde700 1 -- start start 2023-10-06T01:39:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17adde700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe1741595a0 0x7fe1740aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17adde700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe1740b49e0 0x7fe1740aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17adde700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740b0ca0 con 0x7fe1740b07a0 2023-10-06T01:39:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17adde700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740b0f80 con 0x7fe1740b49e0 2023-10-06T01:39:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17adde700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740b1260 con 0x7fe1741595a0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe173fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe1740b49e0 0x7fe1740aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17937b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe1740b07a0 0x7fe1740b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36402/0 (socket says 172.21.15.98:36402) 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe17937b700 1 -- 172.21.15.98:0/3321366775 learned_addr learned my addr 172.21.15.98:0/3321366775 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe178b7a700 1 --2- 172.21.15.98:0/3321366775 >> v2:172.21.15.184:3300/0 conn(0x7fe1741595a0 0x7fe1740aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.958+0000 7fe173fff700 1 -- 172.21.15.98:0/3321366775 >> v2:172.21.15.184:3300/0 conn(0x7fe1741595a0 msgr2=0x7fe1740aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe173fff700 1 --2- 172.21.15.98:0/3321366775 >> v2:172.21.15.184:3300/0 conn(0x7fe1741595a0 0x7fe1740aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe173fff700 1 -- 172.21.15.98:0/3321366775 >> v1:172.21.15.98:6789/0 conn(0x7fe1740b07a0 legacy=0x7fe1740b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe1737fe700 1 -- 172.21.15.98:0/3321366775 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655621589 0 0) 0x7fe1740b0ca0 con 0x7fe1740b07a0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe173fff700 1 -- 172.21.15.98:0/3321366775 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740b1540 con 0x7fe1740b49e0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe173fff700 1 --2- 172.21.15.98:0/3321366775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b49e0 0x7fe1740aaab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe1680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4820704b734fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe1737fe700 1 -- 172.21.15.98:0/3321366775 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011630 con 0x7fe1740b49e0 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe1737fe700 1 -- 172.21.15.98:0/3321366775 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168011790 con 0x7fe1740b49e0 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe1737fe700 1 -- 172.21.15.98:0/3321366775 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011b30 con 0x7fe1740b49e0 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 -- 172.21.15.98:0/3321366775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b49e0 msgr2=0x7fe1740aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 --2- 172.21.15.98:0/3321366775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b49e0 0x7fe1740aaab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe1680091a0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 -- 172.21.15.98:0/3321366775 shutdown_connections 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 --2- 172.21.15.98:0/3321366775 >> v2:172.21.15.184:3300/0 conn(0x7fe1741595a0 0x7fe1740aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 --2- 172.21.15.98:0/3321366775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b49e0 0x7fe1740aaab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 -- 172.21.15.98:0/3321366775 >> 172.21.15.98:0/3321366775 conn(0x7fe174099830 msgr2=0x7fe17406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 -- 172.21.15.98:0/3321366775 shutdown_connections 2023-10-06T01:39:59.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.959+0000 7fe17adde700 1 -- 172.21.15.98:0/3321366775 wait complete. 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.960+0000 7fe17adde700 1 Processor -- start 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 -- start start 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740b49e0 0x7fe174154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1741595a0 0x7fe174155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740cec20 con 0x7fe1740b07a0 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174169b90 con 0x7fe1741595a0 2023-10-06T01:39:59.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe17adde700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe174169e70 con 0x7fe1740b49e0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.961+0000 7fe178b7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34084/0 (socket says 172.21.15.98:34084) 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 -- 172.21.15.98:0/3426740111 learned_addr learned my addr 172.21.15.98:0/3426740111 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe173fff700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740b49e0 0x7fe174154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe17937b700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1741595a0 0x7fe174155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 -- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740b49e0 msgr2=0x7fe174154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740b49e0 0x7fe174154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 -- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1741595a0 msgr2=0x7fe174155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1741595a0 0x7fe174155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 -- 172.21.15.98:0/3426740111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009420 con 0x7fe1740b07a0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe178b7a700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe164009ec0 tx=0x7fe16400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400ccb0 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400f040 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164014480 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17416a150 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.962+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe174150350 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.963+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17416a340 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.963+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe164014830 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.964+0000 7fe170ff9700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.964+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe1640851b0 con 0x7fe1740b07a0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.964+0000 7fe173fff700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:39:59.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.964+0000 7fe173fff700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 0x7fe1600693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe168017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:39:59.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:39:59.968+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe164051fd0 con 0x7fe1740b07a0 2023-10-06T01:40:00.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.154+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe174064ad0 con 0x7fe1740b07a0 2023-10-06T01:40:00.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.155+0000 7fe170ff9700 1 -- 172.21.15.98:0/3426740111 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fe1640555f0 con 0x7fe1740b07a0 2023-10-06T01:40:00.157 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:00.157 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:00.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 msgr2=0x7fe1600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 0x7fe1600693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe168017000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 msgr2=0x7fe17414f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe164009ec0 tx=0x7fe16400b040 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 shutdown_connections 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1741595a0 0x7fe174155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1740b07a0 0x7fe17414f5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 --2- 172.21.15.98:0/3426740111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740b49e0 0x7fe174154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 >> 172.21.15.98:0/3426740111 conn(0x7fe174099830 msgr2=0x7fe1740b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 shutdown_connections 2023-10-06T01:40:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.157+0000 7fe17adde700 1 -- 172.21.15.98:0/3426740111 wait complete. 2023-10-06T01:40:00.171 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T01:40:00.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:00.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.284+0000 7f82fa61c700 1 Processor -- start 2023-10-06T01:40:00.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.284+0000 7f82fa61c700 1 -- start start 2023-10-06T01:40:00.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82fa61c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82fa61c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f82f40c6260 0x7f82f40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82fa61c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40cd610 con 0x7f82f40cd1a0 2023-10-06T01:40:00.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82fa61c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40cc840 con 0x7f82f40c6260 2023-10-06T01:40:00.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82fa61c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40cbc70 con 0x7f82f41594b0 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f9e1b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f82f40c6260 0x7f82f40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58738/0 (socket says 172.21.15.98:58738) 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 -- 172.21.15.98:0/3156473581 learned_addr learned my addr 172.21.15.98:0/3156473581 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 -- 172.21.15.98:0/3156473581 >> v2:172.21.15.184:3300/0 conn(0x7f82f40c6260 msgr2=0x7f82f40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 --2- 172.21.15.98:0/3156473581 >> v2:172.21.15.184:3300/0 conn(0x7f82f40c6260 0x7f82f40cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.285+0000 7f82f961a700 1 -- 172.21.15.98:0/3156473581 >> v1:172.21.15.184:6789/0 conn(0x7f82f40cd1a0 legacy=0x7f82f40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82f961a700 1 -- 172.21.15.98:0/3156473581 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40cbf10 con 0x7f82f41594b0 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82f961a700 1 --2- 172.21.15.98:0/3156473581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009990 tx=0x7f82ec009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a55972b2919b436 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82f3fff700 1 -- 172.21.15.98:0/3156473581 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec009e80 con 0x7f82f41594b0 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82f3fff700 1 -- 172.21.15.98:0/3156473581 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec005c10 con 0x7f82f41594b0 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82f3fff700 1 -- 172.21.15.98:0/3156473581 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00b650 con 0x7f82f41594b0 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 -- 172.21.15.98:0/3156473581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 msgr2=0x7f82f40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 --2- 172.21.15.98:0/3156473581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009990 tx=0x7f82ec009220 comp rx=0 tx=0).stop 2023-10-06T01:40:00.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 -- 172.21.15.98:0/3156473581 shutdown_connections 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 --2- 172.21.15.98:0/3156473581 >> v2:172.21.15.184:3300/0 conn(0x7f82f40c6260 0x7f82f40cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 --2- 172.21.15.98:0/3156473581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40ccc60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.286+0000 7f82fa61c700 1 -- 172.21.15.98:0/3156473581 >> 172.21.15.98:0/3156473581 conn(0x7f82f4099600 msgr2=0x7f82f406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.287+0000 7f82fa61c700 1 -- 172.21.15.98:0/3156473581 shutdown_connections 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.287+0000 7f82fa61c700 1 -- 172.21.15.98:0/3156473581 wait complete. 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.288+0000 7f82fa61c700 1 Processor -- start 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.288+0000 7f82fa61c700 1 -- start start 2023-10-06T01:40:00.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f40c6260 0x7f82f4146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4147370 con 0x7f82f40cd1a0 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c9830 con 0x7f82f40c6260 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82fa61c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c8c20 con 0x7f82f41594b0 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34092/0 (socket says 172.21.15.98:34092) 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 -- 172.21.15.98:0/1798593753 learned_addr learned my addr 172.21.15.98:0/1798593753 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f9e1b700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f961a700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f40c6260 0x7f82f4146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 -- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 msgr2=0x7f82f40b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 -- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f40c6260 msgr2=0x7f82f4146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f40c6260 0x7f82f4146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 -- 172.21.15.98:0/1798593753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec009480 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.289+0000 7f82f8e19700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c570 tx=0x7f82e8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.290+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011590 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.290+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e80116f0 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.290+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40c8f00 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.290+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011ab0 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.290+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f40c94c0 con 0x7f82f40cd1a0 2023-10-06T01:40:00.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.291+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f405f270 con 0x7f82f40cd1a0 2023-10-06T01:40:00.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.291+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82e801b3b0 con 0x7f82f40cd1a0 2023-10-06T01:40:00.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.291+0000 7f82f17fa700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.291+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f82e8083c60 con 0x7f82f40cd1a0 2023-10-06T01:40:00.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.291+0000 7f82f961a700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.292+0000 7f82f961a700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82ec007fd0 tx=0x7f82ec016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.302 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.296+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e8050b30 con 0x7f82f40cd1a0 2023-10-06T01:40:00.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.481+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f40cc1f0 con 0x7f82f40cd1a0 2023-10-06T01:40:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.482+0000 7f82f17fa700 1 -- 172.21.15.98:0/1798593753 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f82e8017070 con 0x7f82f40cd1a0 2023-10-06T01:40:00.484 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:00.484 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 msgr2=0x7f82e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82ec007fd0 tx=0x7f82ec016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 msgr2=0x7f82f40c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c570 tx=0x7f82e8001070 comp rx=0 tx=0).stop 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 shutdown_connections 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f41594b0 0x7f82f40b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f40c6260 0x7f82f4146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.484+0000 7f82fa61c700 1 --2- 172.21.15.98:0/1798593753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82f40cd1a0 0x7f82f40c9cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.485+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 >> 172.21.15.98:0/1798593753 conn(0x7f82f4099600 msgr2=0x7f82f40cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.485+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 shutdown_connections 2023-10-06T01:40:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.485+0000 7f82fa61c700 1 -- 172.21.15.98:0/1798593753 wait complete. 2023-10-06T01:40:00.499 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T01:40:00.500 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T01:40:00.500 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T01:40:00.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T01:40:00.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T01:40:00.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T01:40:00.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T01:40:00.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T01:40:00.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.697+0000 7f88756da700 1 Processor -- start 2023-10-06T01:40:00.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.697+0000 7f88756da700 1 -- start start 2023-10-06T01:40:00.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.697+0000 7f3d3bbdd700 1 Processor -- start 2023-10-06T01:40:00.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f88756da700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f88756da700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88700aa460 0x7f88700aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f88756da700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870068fb0 con 0x7f8870159690 2023-10-06T01:40:00.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f88756da700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88700a9fd0 con 0x7f88700aa460 2023-10-06T01:40:00.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f88756da700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700abfe0 con 0x7f8870068600 2023-10-06T01:40:00.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f3d3bbdd700 1 -- start start 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58766/0 (socket says 172.21.15.98:58766) 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 -- 172.21.15.98:0/1199691016 learned_addr learned my addr 172.21.15.98:0/1199691016 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 -- 172.21.15.98:0/1199691016 >> v2:172.21.15.184:3300/0 conn(0x7f88700aa460 msgr2=0x7f88700aa8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 --2- 172.21.15.98:0/1199691016 >> v2:172.21.15.184:3300/0 conn(0x7f88700aa460 0x7f88700aa8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.698+0000 7f886e7fc700 1 -- 172.21.15.98:0/1199691016 >> v1:172.21.15.98:6789/0 conn(0x7f8870159690 legacy=0x7f88700ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f886e7fc700 1 -- 172.21.15.98:0/1199691016 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700abd00 con 0x7f8870068600 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f886e7fc700 1 --2- 172.21.15.98:0/1199691016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88640099b0 tx=0x7f8864009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c5625d04640fe7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f886dffb700 1 -- 172.21.15.98:0/1199691016 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400b820 con 0x7f8870068600 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f886dffb700 1 -- 172.21.15.98:0/1199691016 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400b980 con 0x7f8870068600 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f886dffb700 1 -- 172.21.15.98:0/1199691016 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400bd20 con 0x7f8870068600 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f3d3bbdd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d341562c0 0x7f3d3415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f3d3bbdd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d34156690 0x7f3d340abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f3d3bbdd700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340c9740 con 0x7f3d34068600 2023-10-06T01:40:00.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f3d3bbdd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3415b040 con 0x7f3d34156690 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.699+0000 7f3d3bbdd700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d340ad850 con 0x7f3d341562c0 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d34156690 0x7f3d340abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d34156690 0x7f3d340abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34108/0 (socket says 172.21.15.98:34108) 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 -- 172.21.15.98:0/781868307 learned_addr learned my addr 172.21.15.98:0/781868307 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f88756da700 1 -- 172.21.15.98:0/1199691016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 msgr2=0x7f8870068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f88756da700 1 --2- 172.21.15.98:0/1199691016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88640099b0 tx=0x7f8864009250 comp rx=0 tx=0).stop 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 -- 172.21.15.98:0/781868307 >> v2:172.21.15.184:3300/0 conn(0x7f3d341562c0 msgr2=0x7f3d3415a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 --2- 172.21.15.98:0/781868307 >> v2:172.21.15.184:3300/0 conn(0x7f3d341562c0 0x7f3d3415a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 -- 172.21.15.98:0/781868307 >> v1:172.21.15.184:6789/0 conn(0x7f3d34068600 legacy=0x7f3d340af980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 -- 172.21.15.98:0/781868307 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3415a7a0 con 0x7f3d34156690 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d39178700 1 --2- 172.21.15.98:0/781868307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d34156690 0x7f3d340abb50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d34070bb0 tx=0x7f3d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e25cea52fdf63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d38977700 1 -- 172.21.15.98:0/781868307 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000b820 con 0x7f3d34156690 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d38977700 1 -- 172.21.15.98:0/781868307 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3000b980 con 0x7f3d34156690 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.700+0000 7f3d38977700 1 -- 172.21.15.98:0/781868307 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000bd20 con 0x7f3d34156690 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 -- 172.21.15.98:0/1199691016 shutdown_connections 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 --2- 172.21.15.98:0/1199691016 >> v2:172.21.15.184:3300/0 conn(0x7f88700aa460 0x7f88700aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 --2- 172.21.15.98:0/1199691016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8870068600 0x7f8870068a70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 -- 172.21.15.98:0/1199691016 >> 172.21.15.98:0/1199691016 conn(0x7f887006de80 msgr2=0x7f887006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f7a04297700 1 Processor -- start 2023-10-06T01:40:00.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f7a04297700 1 -- start start 2023-10-06T01:40:00.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 -- 172.21.15.98:0/1199691016 shutdown_connections 2023-10-06T01:40:00.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f4d1a0a5700 1 Processor -- start 2023-10-06T01:40:00.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f4d1a0a5700 1 -- start start 2023-10-06T01:40:00.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/781868307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d34156690 msgr2=0x7f3d340abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/781868307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d34156690 0x7f3d340abb50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d34070bb0 tx=0x7f3d30009250 comp rx=0 tx=0).stop 2023-10-06T01:40:00.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/781868307 shutdown_connections 2023-10-06T01:40:00.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/781868307 >> v2:172.21.15.184:3300/0 conn(0x7f3d341562c0 0x7f3d3415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/781868307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d34156690 0x7f3d340abb50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/781868307 >> 172.21.15.98:0/781868307 conn(0x7f3d3406de80 msgr2=0x7f3d3406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d1a0a5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d14157460 0x7f4d140ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d1a0a5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157830 0x7f4d140cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d1a0a5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d140cae60 con 0x7f4d140ac480 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d1a0a5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140aeed0 con 0x7f4d14157460 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d1a0a5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140af1b0 con 0x7f4d14157830 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d137fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d14157460 0x7f4d140ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d137fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d14157460 0x7f4d140ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34120/0 (socket says 172.21.15.98:34120) 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d137fe700 1 -- 172.21.15.98:0/3505647129 learned_addr learned my addr 172.21.15.98:0/3505647129 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f4d0bfff700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157830 0x7f4d140cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.701+0000 7f88756da700 1 -- 172.21.15.98:0/1199691016 wait complete. 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d137fe700 1 -- 172.21.15.98:0/3505647129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157830 msgr2=0x7f4d140cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d137fe700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157830 0x7f4d140cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d137fe700 1 -- 172.21.15.98:0/3505647129 >> v1:172.21.15.184:6789/0 conn(0x7f4d140ac480 legacy=0x7f4d14068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d137fe700 1 -- 172.21.15.98:0/3505647129 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140af490 con 0x7f4d14157460 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a04297700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79fc159a60 0x7f79fc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a04297700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc069aa0 0x7f79fc15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a04297700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc15b960 con 0x7f79fc15d920 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a04297700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc15a220 con 0x7f79fc159a60 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a04297700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc069ed0 con 0x7f79fc069aa0 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.702+0000 7f7a02033700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79fc159a60 0x7f79fc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f7a02834700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f79fc15d920 0x7f79fc0c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35812/0 (socket says 172.21.15.98:35812) 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f7a02033700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79fc159a60 0x7f79fc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49792/0 (socket says 172.21.15.98:49792) 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f7a02834700 1 -- 172.21.15.98:0/4166856077 learned_addr learned my addr 172.21.15.98:0/4166856077 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:00.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f88756da700 1 Processor -- start 2023-10-06T01:40:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d137fe700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157460 0x7f4d140ae8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009990 tx=0x7f4d04009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821589581f840c6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f813005d700 1 Processor -- start 2023-10-06T01:40:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f7a01832700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc069aa0 0x7f79fc15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d12ffd700 1 -- 172.21.15.98:0/3505647129 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04009e80 con 0x7f4d14157460 2023-10-06T01:40:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f813005d700 1 -- start start 2023-10-06T01:40:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d12ffd700 1 -- 172.21.15.98:0/3505647129 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d04005c10 con 0x7f4d14157460 2023-10-06T01:40:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f4d12ffd700 1 -- 172.21.15.98:0/3505647129 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0400b650 con 0x7f4d14157460 2023-10-06T01:40:00.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.703+0000 7f88756da700 1 -- start start 2023-10-06T01:40:00.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/3505647129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157460 msgr2=0x7f4d140ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157460 0x7f4d140ae8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009990 tx=0x7f4d04009220 comp rx=0 tx=0).stop 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 0x7f887014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88700aa460 0x7f88701708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8870159690 0x7f887017a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88701568c0 con 0x7f8870068600 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887014f6a0 con 0x7f8870159690 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f88756da700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8870151a40 con 0x7f88700aa460 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f886effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 0x7f887014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f886effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 0x7f887014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34136/0 (socket says 172.21.15.98:34136) 2023-10-06T01:40:00.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f886effd700 1 -- 172.21.15.98:0/885516598 learned_addr learned my addr 172.21.15.98:0/885516598 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f886f7fe700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8870159690 0x7f887017a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210903435 0 0) 0x7f79fc15b960 con 0x7f79fc15d920 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79e8003570 con 0x7f79fc15d920 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3227321218 0 0) 0x7f79e8003570 con 0x7f79fc15d920 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a02033700 1 -- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc069aa0 msgr2=0x7f79fc15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a02033700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc069aa0 0x7f79fc15d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a02033700 1 -- 172.21.15.98:0/4166856077 >> v1:172.21.15.184:6789/0 conn(0x7f79fc15d920 legacy=0x7f79fc0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a02033700 1 -- 172.21.15.98:0/4166856077 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc15dd80 con 0x7f79fc159a60 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f7a02033700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc159a60 0x7f79fc069560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098e0 tx=0x7f79ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c314969aed07f0c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7fc138c6e700 1 Processor -- start 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f813005d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815a940 0x7f81280ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.704+0000 7f813005d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f812815ad10 0x7f81280ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f813005d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8128069560 0x7f81280699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f813005d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81280aeb10 con 0x7f8128069560 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f813005d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8128069f20 con 0x7f812815ad10 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f813005d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812806d5c0 con 0x7f812815a940 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886e7fc700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88700aa460 0x7f88701708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 -- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88700aa460 msgr2=0x7f88701708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88700aa460 0x7f88701708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 -- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8870159690 msgr2=0x7f887017a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4001840 con 0x7f79fc159a60 2023-10-06T01:40:00.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8870159690 0x7f887017a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640094f0 con 0x7f8870068600 2023-10-06T01:40:00.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec011420 con 0x7f79fc159a60 2023-10-06T01:40:00.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f7a01031700 1 -- 172.21.15.98:0/4166856077 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0117e0 con 0x7f79fc159a60 2023-10-06T01:40:00.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7fc138c6e700 1 -- start start 2023-10-06T01:40:00.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f886effd700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 0x7f887014f160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f886000b7e0 tx=0x7f88600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8128069560 0x7f81280699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8128069560 0x7f81280699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34138/0 (socket says 172.21.15.98:34138) 2023-10-06T01:40:00.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 -- 172.21.15.98:0/3194610248 learned_addr learned my addr 172.21.15.98:0/3194610248 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/3505647129 shutdown_connections 2023-10-06T01:40:00.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157460 0x7f4d140ae8f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/3505647129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157830 0x7f4d140cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/3505647129 >> 172.21.15.98:0/3505647129 conn(0x7f4d1406de80 msgr2=0x7f4d1406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812ddf9700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815a940 0x7f81280ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000f510 con 0x7f8870068600 2023-10-06T01:40:00.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000f670 con 0x7f8870068600 2023-10-06T01:40:00.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000fa30 con 0x7f8870068600 2023-10-06T01:40:00.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 -- 172.21.15.98:0/3194610248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815a940 msgr2=0x7f81280ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815a940 0x7f81280ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 -- 172.21.15.98:0/3194610248 >> v2:172.21.15.184:3300/0 conn(0x7f812815ad10 msgr2=0x7f81280ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 --2- 172.21.15.98:0/3194610248 >> v2:172.21.15.184:3300/0 conn(0x7f812815ad10 0x7f81280ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f812e5fa700 1 -- 172.21.15.98:0/3194610248 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280ae290 con 0x7f8128069560 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f812e5fa700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280699e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81280690c0 tx=0x7f8118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b363de6df5bd9cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 -- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc159a60 msgr2=0x7f79fc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc159a60 0x7f79fc069560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098e0 tx=0x7f79ec0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 -- 172.21.15.98:0/4166856077 shutdown_connections 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc159a60 0x7f79fc069560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 --2- 172.21.15.98:0/4166856077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc069aa0 0x7f79fc15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 -- 172.21.15.98:0/4166856077 >> 172.21.15.98:0/4166856077 conn(0x7f79fc0680c0 msgr2=0x7f79fc0cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/3505647129 shutdown_connections 2023-10-06T01:40:00.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.705+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/3505647129 wait complete. 2023-10-06T01:40:00.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f812cdf7700 1 -- 172.21.15.98:0/3194610248 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118009e80 con 0x7f8128069560 2023-10-06T01:40:00.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f812cdf7700 1 -- 172.21.15.98:0/3194610248 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118005c60 con 0x7f8128069560 2023-10-06T01:40:00.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f812cdf7700 1 -- 172.21.15.98:0/3194610248 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811800b610 con 0x7f8128069560 2023-10-06T01:40:00.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8870151d20 con 0x7f8870068600 2023-10-06T01:40:00.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88701522e0 con 0x7f8870068600 2023-10-06T01:40:00.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 -- 172.21.15.98:0/4166856077 shutdown_connections 2023-10-06T01:40:00.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7fc138c6e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc13415d8f0 0x7fc13415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7fc138c6e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1340b1d50 0x7fc1340b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7fc138c6e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340c9750 con 0x7fc134068600 2023-10-06T01:40:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7fc138c6e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340c8ef0 con 0x7fc1340b1d50 2023-10-06T01:40:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7fc138c6e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340b2700 con 0x7fc13415d8f0 2023-10-06T01:40:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.706+0000 7f7a04297700 1 -- 172.21.15.98:0/4166856077 wait complete. 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1340b1d50 0x7fc1340b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1340b1d50 0x7fc1340b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34150/0 (socket says 172.21.15.98:34150) 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 -- 172.21.15.98:0/4114224931 learned_addr learned my addr 172.21.15.98:0/4114224931 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 -- 172.21.15.98:0/3194610248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 msgr2=0x7f81280699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280699e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81280690c0 tx=0x7f8118009250 comp rx=0 tx=0).stop 2023-10-06T01:40:00.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f4d1a0a5700 1 Processor -- start 2023-10-06T01:40:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 -- 172.21.15.98:0/4114224931 >> v2:172.21.15.184:3300/0 conn(0x7fc13415d8f0 msgr2=0x7fc13415dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 --2- 172.21.15.98:0/4114224931 >> v2:172.21.15.184:3300/0 conn(0x7fc13415d8f0 0x7fc13415dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 -- 172.21.15.98:0/4114224931 >> v1:172.21.15.184:6789/0 conn(0x7fc134068600 legacy=0x7fc1341617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 -- 172.21.15.98:0/4114224931 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340b2910 con 0x7fc1340b1d50 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 -- 172.21.15.98:0/3194610248 shutdown_connections 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280699e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 --2- 172.21.15.98:0/3194610248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815a940 0x7f81280ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 --2- 172.21.15.98:0/3194610248 >> v2:172.21.15.184:3300/0 conn(0x7f812815ad10 0x7f81280ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f813005d700 1 -- 172.21.15.98:0/3194610248 >> 172.21.15.98:0/3194610248 conn(0x7f81280680c0 msgr2=0x7f81280cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc131d9b700 1 --2- 172.21.15.98:0/4114224931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1340b1d50 0x7fc1340b21c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc13406dae0 tx=0x7fc124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807a984bdf617b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8870064ad0 con 0x7f8870068600 2023-10-06T01:40:00.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc13159a700 1 -- 172.21.15.98:0/4114224931 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400b820 con 0x7fc1340b1d50 2023-10-06T01:40:00.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc13159a700 1 -- 172.21.15.98:0/4114224931 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12400b980 con 0x7fc1340b1d50 2023-10-06T01:40:00.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.707+0000 7fc13159a700 1 -- 172.21.15.98:0/4114224931 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400bd20 con 0x7fc1340b1d50 2023-10-06T01:40:00.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/781868307 shutdown_connections 2023-10-06T01:40:00.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/781868307 wait complete. 2023-10-06T01:40:00.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f4d1a0a5700 1 -- start start 2023-10-06T01:40:00.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f813005d700 1 -- 172.21.15.98:0/3194610248 shutdown_connections 2023-10-06T01:40:00.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f7a04297700 1 Processor -- start 2023-10-06T01:40:00.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f9d90392700 1 Processor -- start 2023-10-06T01:40:00.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f813005d700 1 -- 172.21.15.98:0/3194610248 wait complete. 2023-10-06T01:40:00.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7fc138c6e700 1 -- 172.21.15.98:0/4114224931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1340b1d50 msgr2=0x7fc1340b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7fc138c6e700 1 --2- 172.21.15.98:0/4114224931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1340b1d50 0x7fc1340b21c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc13406dae0 tx=0x7fc124009250 comp rx=0 tx=0).stop 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7fc138c6e700 1 -- 172.21.15.98:0/4114224931 shutdown_connections 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7fc138c6e700 1 --2- 172.21.15.98:0/4114224931 >> v2:172.21.15.184:3300/0 conn(0x7fc13415d8f0 0x7fc13415dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7fc138c6e700 1 --2- 172.21.15.98:0/4114224931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1340b1d50 0x7fc1340b21c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7fc138c6e700 1 -- 172.21.15.98:0/4114224931 >> 172.21.15.98:0/4114224931 conn(0x7fc13406de80 msgr2=0x7fc13406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f886001a400 con 0x7f8870068600 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f7a04297700 1 -- start start 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc069aa0 0x7f79fc14f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79fc159a60 0x7f79fc0c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 0x7f79fc0c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc17a020 con 0x7f79fc159a60 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc17ae40 con 0x7f79fc069aa0 2023-10-06T01:40:00.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a04297700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc17a230 con 0x7f79fc15d920 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f3d3bbdd700 1 Processor -- start 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f813005d700 1 Processor -- start 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 0x7f4d1413f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157460 0x7f4d14152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157830 0x7f4d141528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d141530d0 con 0x7f4d14157830 2023-10-06T01:40:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d141533b0 con 0x7f4d140ac480 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d1a0a5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14153690 con 0x7f4d14157460 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d137fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 0x7f4d1413f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d137fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 0x7f4d1413f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49834/0 (socket says 172.21.15.98:49834) 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d137fe700 1 -- 172.21.15.98:0/163334137 learned_addr learned my addr 172.21.15.98:0/163334137 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f4d0bfff700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157460 0x7f4d14152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.708+0000 7f9d90392700 1 -- start start 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f9d90392700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d88155a90 0x7f9d880c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a02834700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 0x7f79fc0c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a02834700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 0x7f79fc0c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58822/0 (socket says 172.21.15.98:58822) 2023-10-06T01:40:00.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f7a02834700 1 -- 172.21.15.98:0/2770809333 learned_addr learned my addr 172.21.15.98:0/2770809333 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7fc138c6e700 1 -- 172.21.15.98:0/4114224931 shutdown_connections 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f9d90392700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8806d950 con 0x7f9d88067c10 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f9d90392700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d880c7c00 con 0x7f9d88067840 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.709+0000 7f9d90392700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8806dbf0 con 0x7f9d88155a90 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f9d8e92f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d88155a90 0x7f9d880c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f9d8e92f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d88155a90 0x7f9d880c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58828/0 (socket says 172.21.15.98:58828) 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f9d8e92f700 1 -- 172.21.15.98:0/3385624984 learned_addr learned my addr 172.21.15.98:0/3385624984 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02033700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc069aa0 0x7f79fc14f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f8856ffd700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f885c066fb0 0x7f885c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8860084290 con 0x7f8870068600 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f8856ffd700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7f885c070c00 0x7f885c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f885c073700 con 0x7f885c070c00 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88600092e0 con 0x7f8870068600 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f886f7fe700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7f885c070c00 0x7f885c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02834700 1 -- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc069aa0 msgr2=0x7f79fc14f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02834700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79fc069aa0 0x7f79fc14f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02834700 1 -- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79fc159a60 msgr2=0x7f79fc0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02834700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79fc159a60 0x7f79fc0c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f7a02834700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009420 con 0x7f79fc15d920 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f3d3bbdd700 1 -- start start 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 -- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157460 msgr2=0x7f4d14152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d14157460 0x7f4d14152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 -- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157830 msgr2=0x7f4d141528a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d14157830 0x7f4d141528a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04009480 con 0x7f4d140ac480 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d137fe700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 0x7f4d1413f440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b3c0 tx=0x7f4d04016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527742575 0 0) 0x7f9d880c7c00 con 0x7f9d88067840 2023-10-06T01:40:00.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d74003570 con 0x7f9d88067840 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068600 0x7f3d34154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 0x7f3d340c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34156690 0x7f3d340c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34154ae0 con 0x7f3d341562c0 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340ca350 con 0x7f3d34156690 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3bbdd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34153bf0 con 0x7f3d34068600 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 0x7f3d340c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 0x7f3d340c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34178/0 (socket says 172.21.15.98:34178) 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 -- 172.21.15.98:0/1531502898 learned_addr learned my addr 172.21.15.98:0/1531502898 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d3a17a700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34156690 0x7f3d340c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39979700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068600 0x7f3d34154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 -- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068600 msgr2=0x7f3d34154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068600 0x7f3d34154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 -- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34156690 msgr2=0x7f3d340c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34156690 0x7f3d340c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f3d39178700 1 -- 172.21.15.98:0/1531502898 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300094f0 con 0x7f3d341562c0 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 -- start start 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812815a940 0x7f81280af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815ad10 0x7f81280c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128141090 con 0x7f8128069560 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280bfb40 con 0x7f812815a940 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f813005d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c2900 con 0x7f812815ad10 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f812d5f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812815a940 0x7f81280af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f812e5fa700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815ad10 0x7f81280c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f812ddf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f812ddf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34168/0 (socket says 172.21.15.98:34168) 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f812ddf9700 1 -- 172.21.15.98:0/1466199681 learned_addr learned my addr 172.21.15.98:0/1466199681 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f886e7fc700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f885c066fb0 0x7f885c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04011ae0 con 0x7f4d140ac480 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d04020070 con 0x7f4d140ac480 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04011e70 con 0x7f4d140ac480 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891285176 0 0) 0x7f9d8806d950 con 0x7f9d88067c10 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d880c7c00 con 0x7f9d88067c10 2023-10-06T01:40:00.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2606215201 0 0) 0x7f9d74003570 con 0x7f9d88067840 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d88155a90 msgr2=0x7f9d880c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d88155a90 0x7f9d880c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 >> v1:172.21.15.98:6789/0 conn(0x7f9d88067c10 legacy=0x7f9d880af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8806ded0 con 0x7f9d88067840 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f9d78001470 con 0x7f9d88067840 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 >> v1:172.21.15.184:6789/0 conn(0x7f9d88067840 legacy=0x7f9d88155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d74003bf0 0x7f9d74003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74004810 0x7f9d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d740051c0 0x7f9d74010070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8806dbf0 con 0x7f9d74003bf0 2023-10-06T01:40:00.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740105b0 con 0x7f9d74004810 2023-10-06T01:40:00.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740106f0 con 0x7f9d740051c0 2023-10-06T01:40:00.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.710+0000 7fc138c6e700 1 -- 172.21.15.98:0/4114224931 wait complete. 2023-10-06T01:40:00.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7fc138c6e700 1 Processor -- start 2023-10-06T01:40:00.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.711+0000 7f886f7fe700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7f885c070c00 0x7f885c073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 -- 172.21.15.98:0/1466199681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815ad10 msgr2=0x7f81280c17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815ad10 0x7f81280c17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 -- 172.21.15.98:0/1466199681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812815a940 msgr2=0x7f81280af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812815a940 0x7f81280af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180094d0 con 0x7f8128069560 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812e5fa700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812815ad10 0x7f81280c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:00.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74004810 0x7f9d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7fc138c6e700 1 -- start start 2023-10-06T01:40:00.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8e12e700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d74003bf0 0x7f9d74003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8e92f700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d740051c0 0x7f9d74010070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 -- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d740051c0 msgr2=0x7f9d74010070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d740051c0 0x7f9d74010070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 -- 172.21.15.98:0/3385624984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d74003bf0 msgr2=0x7f9d74003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d74003bf0 0x7f9d74003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 -- 172.21.15.98:0/3385624984 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8806ded0 con 0x7f9d74004810 2023-10-06T01:40:00.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f812ddf9700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280cb590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8120009ef0 tx=0x7f812000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f886e7fc700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f885c066fb0 0x7f885c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88640099b0 tx=0x7f8864016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f7a02834700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 0x7f79fc0c3670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79f400cf40 tx=0x7f79f4001da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74004810 0x7f9d74004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d80002140 tx=0x7f9d800017c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c16ad92b96075ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000cb70 con 0x7f8128069560 2023-10-06T01:40:00.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81280c0350 con 0x7f8128069560 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812000f040 con 0x7f8128069560 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120014480 con 0x7f8128069560 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400acf0 con 0x7f79fc15d920 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f400ae90 con 0x7f79fc15d920 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.712+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc17a510 con 0x7f79fc15d920 2023-10-06T01:40:00.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc17aad0 con 0x7f79fc15d920 2023-10-06T01:40:00.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81280c0070 con 0x7f8128069560 2023-10-06T01:40:00.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d14153970 con 0x7f4d140ac480 2023-10-06T01:40:00.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1413f980 con 0x7f4d140ac480 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== osd.2 v2:172.21.15.98:6824/120679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f885c073700 con 0x7f885c070c00 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc134068600 0x7fc1340ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 0x7fc134179e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415d8f0 0x7fc13417a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340adb50 con 0x7fc134068600 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340ade30 con 0x7fc13415d8f0 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc138c6e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340ae110 con 0x7fc1340b1d50 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 0x7fc134179e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 0x7fc134179e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58860/0 (socket says 172.21.15.98:58860) 2023-10-06T01:40:00.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc132d9d700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415d8f0 0x7fc13417a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 -- 172.21.15.98:0/1130114933 learned_addr learned my addr 172.21.15.98:0/1130114933 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40073a0 con 0x7f79fc15d920 2023-10-06T01:40:00.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 -- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415d8f0 msgr2=0x7fc13417a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415d8f0 0x7fc13417a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 -- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc134068600 msgr2=0x7fc1340ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc134068600 0x7fc1340ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7fc131d9b700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240094f0 con 0x7fc1340b1d50 2023-10-06T01:40:00.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d14153520 con 0x7f4d140ac480 2023-10-06T01:40:00.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.713+0000 7f3d39178700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 0x7f3d340c98d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d300090b0 tx=0x7f3d30018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7fc131d9b700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 0x7fc134179e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc124003080 tx=0x7fc124016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79e4000fd0 con 0x7f79fc15d920 2023-10-06T01:40:00.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30010e70 con 0x7f3d341562c0 2023-10-06T01:40:00.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d30018630 con 0x7f3d341562c0 2023-10-06T01:40:00.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300189f0 con 0x7f3d341562c0 2023-10-06T01:40:00.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d34153ed0 con 0x7f3d341562c0 2023-10-06T01:40:00.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d34154490 con 0x7f3d341562c0 2023-10-06T01:40:00.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79f4007750 con 0x7f79fc15d920 2023-10-06T01:40:00.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8128064ad0 con 0x7f8128069560 2023-10-06T01:40:00.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124011d70 con 0x7fc1340b1d50 2023-10-06T01:40:00.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc124020070 con 0x7fc1340b1d50 2023-10-06T01:40:00.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12401b6a0 con 0x7fc1340b1d50 2023-10-06T01:40:00.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f79e0066fb0 0x7f79e0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f79f4084c30 con 0x7f79fc15d920 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f79e0070c00 0x7f79e0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80001ad0 con 0x7f9d74004810 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8000c940 con 0x7f9d74004810 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.714+0000 7f9d8d12c700 1 -- 172.21.15.98:0/3385624984 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000cd00 con 0x7f9d74004810 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79e0073700 con 0x7f79e0070c00 2023-10-06T01:40:00.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.715+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79f4051ae0 con 0x7f79fc15d920 2023-10-06T01:40:00.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f9d90392700 1 -- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74004810 msgr2=0x7f9d74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f9d90392700 1 --2- 172.21.15.98:0/3385624984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74004810 0x7f9d74004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d80002140 tx=0x7f9d800017c0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d04022030 con 0x7f4d140ac480 2023-10-06T01:40:00.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d300233f0 con 0x7f3d341562c0 2023-10-06T01:40:00.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f4d0b7fe700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4cf4067080 0x7f4cf4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4d04086b80 con 0x7f4d140ac480 2023-10-06T01:40:00.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f7a02033700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f79e0066fb0 0x7f79e0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f7a01832700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f79e0070c00 0x7f79e0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.716+0000 7f7a02033700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f79e0066fb0 0x7f79e0069460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098e0 tx=0x7f79ec016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f4d0b7fe700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f4cf4070d50 0x7f4cf4073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cf4073850 con 0x7f4cf4070d50 2023-10-06T01:40:00.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d04086eb0 con 0x7f4d140ac480 2023-10-06T01:40:00.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f9d8e92f700 1 -- 172.21.15.98:0/3385624984 reap_dead start 2023-10-06T01:40:00.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7fb89d332700 1 Processor -- start 2023-10-06T01:40:00.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7fb89d332700 1 -- start start 2023-10-06T01:40:00.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f3d29ffb700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d18066f30 0x7f3d180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3d30014070 con 0x7f3d341562c0 2023-10-06T01:40:00.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f3d39979700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d18066f30 0x7f3d180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f9d90392700 1 -- 172.21.15.98:0/3385624984 shutdown_connections 2023-10-06T01:40:00.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f9d90392700 1 -- 172.21.15.98:0/3385624984 >> 172.21.15.98:0/3385624984 conn(0x7f9d880cc730 msgr2=0x7f9d880ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f4d0bfff700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4cf4067080 0x7f4cf4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f9d90392700 1 -- 172.21.15.98:0/3385624984 shutdown_connections 2023-10-06T01:40:00.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f9d90392700 1 -- 172.21.15.98:0/3385624984 wait complete. 2023-10-06T01:40:00.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f3d39979700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d18066f30 0x7f3d180693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006fd0 tx=0x7f3d24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81200147f0 con 0x7f8128069560 2023-10-06T01:40:00.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f810c066f30 0x7f810c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8120085ae0 con 0x7f8128069560 2023-10-06T01:40:00.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f810c070c00 0x7f810c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f810c073700 con 0x7f810c070c00 2023-10-06T01:40:00.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8120085e10 con 0x7f8128069560 2023-10-06T01:40:00.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f7a01832700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f79e0070c00 0x7f79e0073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7f4d0bfff700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4cf4067080 0x7f4cf4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d14155bc0 tx=0x7f4cfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.717+0000 7f812d5f8700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f810c066f30 0x7f810c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7f812d5f8700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f810c066f30 0x7f810c0693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81280b1ef0 tx=0x7f81240090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb89d332700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb898069c70 0x7fb8980aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb89d332700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8980c7e80 con 0x7fb898160e30 2023-10-06T01:40:00.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb896ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb898160e30 0x7fb8980c8310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36430/0 (socket says 172.21.15.98:36430) 2023-10-06T01:40:00.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb896ffd700 1 -- 172.21.15.98:0/233697132 learned_addr learned my addr 172.21.15.98:0/233697132 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:00.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb898161680 con 0x7fb898161200 2023-10-06T01:40:00.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f79e0073700 con 0x7f79e0070c00 2023-10-06T01:40:00.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980c8a20 con 0x7fb898069c70 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb8977fe700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb898069c70 0x7fb8980aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034100538 0 0) 0x7fb8980c7e80 con 0x7fb898160e30 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb884003570 con 0x7fb898160e30 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7f3d3414e830 0x7f3d3414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d34064ad0 con 0x7f3d3414e830 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1340ae3f0 con 0x7fc1340b1d50 2023-10-06T01:40:00.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1340aebf0 con 0x7fc1340b1d50 2023-10-06T01:40:00.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f3d3a17a700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7f3d3414e830 0x7f3d3414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f4d13fff700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f4cf4070d50 0x7f4cf4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f9d90392700 1 Processor -- start 2023-10-06T01:40:00.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f9d90392700 1 -- start start 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317933897 0 0) 0x7fb898161680 con 0x7fb898161200 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8980c7e80 con 0x7fb898161200 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190150184 0 0) 0x7fb884003570 con 0x7fb898160e30 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb898069c70 msgr2=0x7fb8980aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb898069c70 0x7fb8980aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 >> v1:172.21.15.184:6789/0 conn(0x7fb898161200 legacy=0x7fb898069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980c7270 con 0x7fb898160e30 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fb8900017f0 con 0x7fb898160e30 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 >> v1:172.21.15.98:6789/0 conn(0x7fb898160e30 legacy=0x7fb8980c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb884003c20 0x7fb884004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb884004810 0x7fb884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8840052e0 0x7fb88400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980c8a20 con 0x7fb884003c20 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88400c1e0 con 0x7fb8840052e0 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88400c3a0 con 0x7fb884004810 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb8977fe700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb898069c70 0x7fb8980aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:00.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7fb896ffd700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb884003c20 0x7fb884004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f4d13fff700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f4cf4070d50 0x7f4cf4073180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb8977fe700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8840052e0 0x7fb88400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.718+0000 7f812e5fa700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f810c070c00 0x7f810c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1340adfa0 con 0x7fc1340b1d50 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb896ffd700 1 -- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb884004810 msgr2=0x7fb884004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb8967fc700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb884004810 0x7fb884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb896ffd700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb884004810 0x7fb884004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb896ffd700 1 -- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8840052e0 msgr2=0x7fb88400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb896ffd700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8840052e0 0x7fb88400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb896ffd700 1 -- 172.21.15.98:0/233697132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980c7270 con 0x7fb884003c20 2023-10-06T01:40:00.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f812e5fa700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f810c070c00 0x7f810c073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7fb8967fc700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb884004810 0x7fb884004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:00.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.719+0000 7f3d3a17a700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7f3d3414e830 0x7f3d3414ec00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 0x7f9d880c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d88153100 0x7f9d880c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d880c4c50 0x7f9d880ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880c67b0 con 0x7f9d880c4c50 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880c5de0 con 0x7f9d88153100 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d90392700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880c51d0 con 0x7f9d74004810 2023-10-06T01:40:00.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3d34064ad0 con 0x7f3d3414e830 2023-10-06T01:40:00.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f810c073700 con 0x7f810c070c00 2023-10-06T01:40:00.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7fb896ffd700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb884003c20 0x7fb884004010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb89000a960 tx=0x7fb89000ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cd3047ad05d95d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4cf4073850 con 0x7f4cf4070d50 2023-10-06T01:40:00.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d8e12e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 0x7f9d880c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d8e92f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d880c4c50 0x7f9d880ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d8e12e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 0x7f9d880c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58876/0 (socket says 172.21.15.98:58876) 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d8e12e700 1 -- 172.21.15.98:0/1192316587 learned_addr learned my addr 172.21.15.98:0/1192316587 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.720+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d88153100 0x7f9d880c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 -- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d88153100 msgr2=0x7f9d880c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d88153100 0x7f9d880c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 -- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d880c4c50 msgr2=0x7f9d880ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d880c4c50 0x7f9d880ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880c54b0 con 0x7f9d74004810 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e92f700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d880c4c50 0x7f9d880ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d8e12e700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 0x7f9d880c6270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800fd80 tx=0x7f9d78007830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800fee0 con 0x7f9d74004810 2023-10-06T01:40:00.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7800db30 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78025480 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d880c5790 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.721+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d880b2cb0 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.722+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d88064ad0 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d780258b0 con 0x7f9d74004810 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.722+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890007110 con 0x7fb884003c20 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.722+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890007270 con 0x7fb884003c20 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.722+0000 7fb895ffb700 1 -- 172.21.15.98:0/233697132 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890007610 con 0x7fb884003c20 2023-10-06T01:40:00.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb884003c20 msgr2=0x7fb884004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fb89d332700 1 --2- 172.21.15.98:0/233697132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb884003c20 0x7fb884004010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb89000a960 tx=0x7fb89000ac70 comp rx=0 tx=0).stop 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.722+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc124022030 con 0x7fc1340b1d50 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc1227fc700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc114066fb0 0x7fc114069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc124086900 con 0x7fc1340b1d50 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc1227fc700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fc114070c80 0x7fc1140730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc114073780 con 0x7fc114070c80 2023-10-06T01:40:00.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc12408b220 con 0x7fc1340b1d50 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.723+0000 7fc132d9d700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fc114070c80 0x7fc1140730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7fc132d9d700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fc114070c80 0x7fc1140730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7fc13259c700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc114066fb0 0x7fc114069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d7e7fc700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d6c066f30 0x7f9d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9d780963f0 con 0x7f9d74004810 2023-10-06T01:40:00.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d7e7fc700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7f9d6c070c00 0x7f9d6c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d6c073700 con 0x7f9d6c070c00 2023-10-06T01:40:00.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d780989d0 con 0x7f9d74004810 2023-10-06T01:40:00.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d6c066f30 0x7f9d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7fc13259c700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc114066fb0 0x7fc114069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00ad10 tx=0x7fc12c009370 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7f9d8e92f700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7f9d6c070c00 0x7f9d6c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.724+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc114073780 con 0x7fc114070c80 2023-10-06T01:40:00.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.725+0000 7f9d8d92d700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d6c066f30 0x7f9d6c0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d880b32d0 tx=0x7f9d80016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.786 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.726+0000 7fb8977fe700 1 -- 172.21.15.98:0/233697132 reap_dead start 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.726+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 shutdown_connections 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.726+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 >> 172.21.15.98:0/233697132 conn(0x7fb8980680c0 msgr2=0x7fb8980cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.726+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 shutdown_connections 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.726+0000 7fb89d332700 1 -- 172.21.15.98:0/233697132 wait complete. 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.728+0000 7fb89d332700 1 Processor -- start 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.728+0000 7fb89d332700 1 -- start start 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8980c4840 0x7fb89816e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb89816e780 0x7fb8980c0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 0x7fb8980cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816f1c0 con 0x7fb8980c4840 2023-10-06T01:40:00.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816ec00 con 0x7fb8980c09d0 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb89d332700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816eee0 con 0x7fb89816e780 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb8967fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb89816e780 0x7fb8980c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb8977fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 0x7fb8980cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb8977fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 0x7fb8980cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49908/0 (socket says 172.21.15.98:49908) 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.729+0000 7fb8977fe700 1 -- 172.21.15.98:0/3122654350 learned_addr learned my addr 172.21.15.98:0/3122654350 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 -- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb89816e780 msgr2=0x7fb8980c0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb89816e780 0x7fb8980c0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 -- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8980c4840 msgr2=0x7fb89816e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8980c4840 0x7fb89816e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 -- 172.21.15.98:0/3122654350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980bfc90 con 0x7fb8980c09d0 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb8977fe700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 0x7fb8980cadd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb89806a040 tx=0x7fb880008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880013450 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8800135b0 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880013970 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8980bff70 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89815ea40 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.733+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb880013ce0 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.733+0000 7fb88effd700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb884068b40 0x7fb88406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.733+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb880084910 con 0x7fb8980c09d0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.733+0000 7fb896ffd700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb884068b40 0x7fb88406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.734+0000 7fb896ffd700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb884068b40 0x7fb88406aff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb89000acb0 tx=0x7fb890015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.730+0000 7f9d8e92f700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7f9d6c070c00 0x7f9d6c073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.737+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9d6c073700 con 0x7f9d6c070c00 2023-10-06T01:40:00.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.738+0000 7fb89d332700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fb87c0015f0 0x7fb87c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:00.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.738+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb87c006b90 con 0x7fb87c0015f0 2023-10-06T01:40:00.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.741+0000 7fb8967fc700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fb87c0015f0 0x7fb87c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:00.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.741+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d14064ad0 con 0x7f4cf4070d50 2023-10-06T01:40:00.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.741+0000 7f4d0b7fe700 1 -- 172.21.15.98:0/163334137 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4d14064ad0 con 0x7f4cf4070d50 2023-10-06T01:40:00.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.741+0000 7fb8967fc700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fb87c0015f0 0x7fb87c003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f4cf4070d50 msgr2=0x7f4cf4073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f4cf4070d50 0x7f4cf4073180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4cf4067080 msgr2=0x7f4cf4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4cf4067080 0x7f4cf4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d14155bc0 tx=0x7f4cfc008040 comp rx=0 tx=0).stop 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 msgr2=0x7f4d1413f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d1a0a5700 1 --2- 172.21.15.98:0/163334137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d140ac480 0x7f4d1413f440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b3c0 tx=0x7f4d04016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7f4d13fff700 1 -- 172.21.15.98:0/163334137 reap_dead start 2023-10-06T01:40:00.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.742+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== osd.6 v2:172.21.15.184:6800/120875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb87c006b90 con 0x7fb87c0015f0 2023-10-06T01:40:00.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.743+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 shutdown_connections 2023-10-06T01:40:00.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.743+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 >> 172.21.15.98:0/163334137 conn(0x7f4d1406de80 msgr2=0x7f4d14154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.743+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f887006e620 con 0x7f885c070c00 2023-10-06T01:40:00.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.744+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 shutdown_connections 2023-10-06T01:40:00.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.745+0000 7f4d1a0a5700 1 -- 172.21.15.98:0/163334137 wait complete. 2023-10-06T01:40:00.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.745+0000 7f8856ffd700 1 -- 172.21.15.98:0/885516598 <== osd.2 v2:172.21.15.98:6824/120679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f887006e620 con 0x7f885c070c00 2023-10-06T01:40:00.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.745+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7f885c070c00 msgr2=0x7f885c073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.745+0000 7f88756da700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7f885c070c00 0x7f885c073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f885c066fb0 msgr2=0x7f885c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f885c066fb0 0x7f885c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88640099b0 tx=0x7f8864016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 msgr2=0x7f887014f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 --2- 172.21.15.98:0/885516598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8870068600 0x7f887014f160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f886000b7e0 tx=0x7f88600091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f886f7fe700 1 -- 172.21.15.98:0/885516598 reap_dead start 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 shutdown_connections 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.746+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 >> 172.21.15.98:0/885516598 conn(0x7f887006de80 msgr2=0x7f88700b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.748+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 shutdown_connections 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.748+0000 7f88756da700 1 -- 172.21.15.98:0/885516598 wait complete. 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.750+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d3404bb80 con 0x7f3d3414e830 2023-10-06T01:40:00.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.750+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81280ae550 con 0x7f810c070c00 2023-10-06T01:40:00.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.750+0000 7f3d29ffb700 1 -- 172.21.15.98:0/1531502898 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d3404bb80 con 0x7f3d3414e830 2023-10-06T01:40:00.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.751+0000 7f811dffb700 1 -- 172.21.15.98:0/1466199681 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f81280ae550 con 0x7f810c070c00 2023-10-06T01:40:00.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.752+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7f3d3414e830 msgr2=0x7f3d3414ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.752+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7f3d3414e830 0x7f3d3414ec00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.752+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f810c070c00 msgr2=0x7f810c073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.752+0000 7f813005d700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f810c070c00 0x7f810c073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.752+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79e4002d30 con 0x7f79e0070c00 2023-10-06T01:40:00.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f79f27fc700 1 -- 172.21.15.98:0/2770809333 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79e4002d30 con 0x7f79e0070c00 2023-10-06T01:40:00.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f79e0070c00 msgr2=0x7f79e0073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f79e0070c00 0x7f79e0073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f79e0066fb0 msgr2=0x7f79e0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f79e0066fb0 0x7f79e0069460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098e0 tx=0x7f79ec016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 msgr2=0x7f79fc0c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 --2- 172.21.15.98:0/2770809333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79fc15d920 0x7f79fc0c3670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79f400cf40 tx=0x7f79f4001da0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a02834700 1 -- 172.21.15.98:0/2770809333 reap_dead start 2023-10-06T01:40:00.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1340aede0 con 0x7fc114070c80 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 shutdown_connections 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 >> 172.21.15.98:0/2770809333 conn(0x7f79fc0680c0 msgr2=0x7f79fc0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 shutdown_connections 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.753+0000 7f7a04297700 1 -- 172.21.15.98:0/2770809333 wait complete. 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f810c066f30 msgr2=0x7f810c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f810c066f30 0x7f810c0693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81280b1ef0 tx=0x7f81240090f0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 msgr2=0x7f81280cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 --2- 172.21.15.98:0/1466199681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8128069560 0x7f81280cb590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8120009ef0 tx=0x7f812000b040 comp rx=0 tx=0).stop 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7fc1227fc700 1 -- 172.21.15.98:0/1130114933 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1340aede0 con 0x7fc114070c80 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f812e5fa700 1 -- 172.21.15.98:0/1466199681 reap_dead start 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 shutdown_connections 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 >> 172.21.15.98:0/1466199681 conn(0x7f81280680c0 msgr2=0x7f812806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 shutdown_connections 2023-10-06T01:40:00.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.754+0000 7f813005d700 1 -- 172.21.15.98:0/1466199681 wait complete. 2023-10-06T01:40:00.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.758+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d8806e1b0 con 0x7f9d6c070c00 2023-10-06T01:40:00.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.759+0000 7f9d7e7fc700 1 -- 172.21.15.98:0/1192316587 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d8806e1b0 con 0x7f9d6c070c00 2023-10-06T01:40:00.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7f9d6c070c00 msgr2=0x7f9d6c073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7f9d6c070c00 0x7f9d6c073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d6c066f30 msgr2=0x7f9d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d6c066f30 0x7f9d6c0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d880b32d0 tx=0x7f9d80016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 msgr2=0x7f9d880c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d90392700 1 --2- 172.21.15.98:0/1192316587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74004810 0x7f9d880c6270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800fd80 tx=0x7f9d78007830 comp rx=0 tx=0).stop 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.760+0000 7f9d8e92f700 1 -- 172.21.15.98:0/1192316587 reap_dead start 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.761+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 shutdown_connections 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.761+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 >> 172.21.15.98:0/1192316587 conn(0x7f9d880cc730 msgr2=0x7f9d8806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.761+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 shutdown_connections 2023-10-06T01:40:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.761+0000 7f9d90392700 1 -- 172.21.15.98:0/1192316587 wait complete. 2023-10-06T01:40:00.806 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:00.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.764+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fc114070c80 msgr2=0x7fc1140730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.764+0000 7fc138c6e700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fc114070c80 0x7fc1140730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.756+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d18066f30 msgr2=0x7f3d180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.756+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d18066f30 0x7f3d180693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006fd0 tx=0x7f3d24008040 comp rx=0 tx=0).stop 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.756+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 msgr2=0x7f3d340c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.756+0000 7f3d3bbdd700 1 --2- 172.21.15.98:0/1531502898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d341562c0 0x7f3d340c98d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d300090b0 tx=0x7f3d30018000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7f3d3a17a700 1 -- 172.21.15.98:0/1531502898 reap_dead start 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 shutdown_connections 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 >> 172.21.15.98:0/1531502898 conn(0x7f3d3406de80 msgr2=0x7f3d340b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc114066fb0 msgr2=0x7fc114069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc114066fb0 0x7fc114069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00ad10 tx=0x7fc12c009370 comp rx=0 tx=0).stop 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 msgr2=0x7fc134179e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 --2- 172.21.15.98:0/1130114933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340b1d50 0x7fc134179e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc124003080 tx=0x7fc124016000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc132d9d700 1 -- 172.21.15.98:0/1130114933 reap_dead start 2023-10-06T01:40:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 shutdown_connections 2023-10-06T01:40:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 shutdown_connections 2023-10-06T01:40:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.765+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 >> 172.21.15.98:0/1130114933 conn(0x7fc13406de80 msgr2=0x7fc1340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.767+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb87c005c80 con 0x7fb87c0015f0 2023-10-06T01:40:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.767+0000 7fb88effd700 1 -- 172.21.15.98:0/3122654350 <== osd.6 v2:172.21.15.184:6800/120875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb87c005c80 con 0x7fb87c0015f0 2023-10-06T01:40:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fb87c0015f0 msgr2=0x7fb87c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fb87c0015f0 0x7fb87c003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb884068b40 msgr2=0x7fb88406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb884068b40 0x7fb88406aff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb89000acb0 tx=0x7fb890015000 comp rx=0 tx=0).stop 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 msgr2=0x7fb8980cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb89d332700 1 --2- 172.21.15.98:0/3122654350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8980c09d0 0x7fb8980cadd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb89806a040 tx=0x7fb880008040 comp rx=0 tx=0).stop 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.768+0000 7fb8977fe700 1 -- 172.21.15.98:0/3122654350 reap_dead start 2023-10-06T01:40:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.769+0000 7f3d3bbdd700 1 -- 172.21.15.98:0/1531502898 wait complete. 2023-10-06T01:40:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.769+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 shutdown_connections 2023-10-06T01:40:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.769+0000 7fc138c6e700 1 -- 172.21.15.98:0/1130114933 wait complete. 2023-10-06T01:40:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.772+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 shutdown_connections 2023-10-06T01:40:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.772+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 >> 172.21.15.98:0/3122654350 conn(0x7fb8980680c0 msgr2=0x7fb8980b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:00.813 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:00.813 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T01:40:00.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.776+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 shutdown_connections 2023-10-06T01:40:00.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:00.776+0000 7fb89d332700 1 -- 172.21.15.98:0/3122654350 wait complete. 2023-10-06T01:40:00.814 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:00.815 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T01:40:00.815 INFO:teuthology.orchestra.run.smithi098.stdout:55834574857 2023-10-06T01:40:00.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T01:40:00.816 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:00.817 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T01:40:00.817 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:00.818 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T01:40:00.818 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:00.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T01:40:00.819 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:00.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T01:40:01.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.049+0000 7f785da2f700 1 Processor -- start 2023-10-06T01:40:01.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.049+0000 7f785da2f700 1 -- start start 2023-10-06T01:40:01.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f5ca740f700 1 Processor -- start 2023-10-06T01:40:01.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f785da2f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7858155860 0x7f78580ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f785da2f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f785da2f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580c7e10 con 0x7f78580c82a0 2023-10-06T01:40:01.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f785da2f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580aea00 con 0x7f7858155860 2023-10-06T01:40:01.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f785da2f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c7200 con 0x7f7858069c70 2023-10-06T01:40:01.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.050+0000 7f5ca740f700 1 -- start start 2023-10-06T01:40:01.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f2b8abfd700 1 Processor -- start 2023-10-06T01:40:01.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f2b8abfd700 1 -- start start 2023-10-06T01:40:01.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca740f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5ca00c82a0 0x7f5ca0155860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca740f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 0x7f5ca0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca740f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca00c7e10 con 0x7f5ca00cbc60 2023-10-06T01:40:01.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca740f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca00acce0 con 0x7f5ca00c82a0 2023-10-06T01:40:01.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca740f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0155da0 con 0x7f5ca006e410 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f78567fc700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f78580c82a0 0x7f7858069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35842/0 (socket says 172.21.15.98:35842) 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f78567fc700 1 -- 172.21.15.98:0/302760013 learned_addr learned my addr 172.21.15.98:0/302760013 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca49aa700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5ca00cbc60 0x7f5ca00ae4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36446/0 (socket says 172.21.15.98:36446) 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca49aa700 1 -- 172.21.15.98:0/1472202006 learned_addr learned my addr 172.21.15.98:0/1472202006 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/1472202006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 0x7f5ca0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f5ca51ab700 1 --2- 172.21.15.98:0/1472202006 >> v2:172.21.15.98:3300/0 conn(0x7f5ca00c82a0 0x7f5ca0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.051+0000 7f78577fe700 1 --2- 172.21.15.98:0/302760013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5ca59ac700 1 -- 172.21.15.98:0/1472202006 >> v2:172.21.15.98:3300/0 conn(0x7f5ca00c82a0 msgr2=0x7f5ca0155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/1472202006 >> v2:172.21.15.98:3300/0 conn(0x7f5ca00c82a0 0x7f5ca0155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5ca59ac700 1 -- 172.21.15.98:0/1472202006 >> v1:172.21.15.98:6789/0 conn(0x7f5ca00cbc60 legacy=0x7f5ca00ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5ca59ac700 1 -- 172.21.15.98:0/1472202006 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00c7200 con 0x7f5ca006e410 2023-10-06T01:40:01.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/1472202006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 0x7f5ca0069560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006fd0 tx=0x7f5c9c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd25df608e440440 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8abfd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b840cbc60 0x7f2b841556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8abfd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 0x7f2b84069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8abfd700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840c7e10 con 0x7f2b840c82a0 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8abfd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b840acce0 con 0x7f2b8406e410 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8abfd700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b840c74e0 con 0x7f2b840cbc60 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8919a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 0x7f2b84069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f20af284700 1 Processor -- start 2023-10-06T01:40:01.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8919a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 0x7f2b84069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34236/0 (socket says 172.21.15.98:34236) 2023-10-06T01:40:01.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b8919a700 1 -- 172.21.15.98:0/1545827479 learned_addr learned my addr 172.21.15.98:0/1545827479 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f2b83fff700 1 --2- 172.21.15.98:0/1545827479 >> v2:172.21.15.184:3300/0 conn(0x7f2b840cbc60 0x7f2b841556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5c97fff700 1 -- 172.21.15.98:0/1472202006 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c005c60 con 0x7f5ca006e410 2023-10-06T01:40:01.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5c97fff700 1 -- 172.21.15.98:0/1472202006 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c005dc0 con 0x7f5ca006e410 2023-10-06T01:40:01.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f5c97fff700 1 -- 172.21.15.98:0/1472202006 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c006280 con 0x7f5ca006e410 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b83fff700 1 -- 172.21.15.98:0/1545827479 >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 msgr2=0x7f2b84069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b83fff700 1 --2- 172.21.15.98:0/1545827479 >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 0x7f2b84069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b83fff700 1 -- 172.21.15.98:0/1545827479 >> v1:172.21.15.98:6789/0 conn(0x7f2b840c82a0 legacy=0x7f2b840ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b83fff700 1 -- 172.21.15.98:0/1545827479 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840c77c0 con 0x7f2b840cbc60 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f78577fe700 1 -- 172.21.15.98:0/302760013 >> v2:172.21.15.184:3300/0 conn(0x7f7858155860 msgr2=0x7f78580ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f78577fe700 1 --2- 172.21.15.98:0/302760013 >> v2:172.21.15.184:3300/0 conn(0x7f7858155860 0x7f78580ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f78577fe700 1 -- 172.21.15.98:0/302760013 >> v1:172.21.15.184:6789/0 conn(0x7f78580c82a0 legacy=0x7f7858069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f78577fe700 1 -- 172.21.15.98:0/302760013 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580c74e0 con 0x7f7858069c70 2023-10-06T01:40:01.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7f20af284700 1 -- start start 2023-10-06T01:40:01.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b83fff700 1 --2- 172.21.15.98:0/1545827479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840cbc60 0x7f2b841556a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b840ab9a0 tx=0x7f2b740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569bcd15c2fce9cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7fb3bfeec700 1 Processor -- start 2023-10-06T01:40:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.052+0000 7fb3bfeec700 1 -- start start 2023-10-06T01:40:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f78577fe700 1 --2- 172.21.15.98:0/302760013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580abcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f784c00aa70 tx=0x7f784c00a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7be9b0b06893fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f18f2d85700 1 Processor -- start 2023-10-06T01:40:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f18f2d85700 1 -- start start 2023-10-06T01:40:01.065 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b837fe700 1 -- 172.21.15.98:0/1545827479 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74011630 con 0x7f2b840cbc60 2023-10-06T01:40:01.065 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.053+0000 7f2b837fe700 1 -- 172.21.15.98:0/1545827479 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74011790 con 0x7f2b840cbc60 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20a80b2390 0x7f20a80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a8162370 0x7f20a80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20a81617c0 0x7f20a8161c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a815e340 con 0x7f20a80b2390 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a8162180 con 0x7f20a81617c0 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20af284700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a815e170 con 0x7f20a8162370 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20a80b2390 0x7f20a80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20a80b2390 0x7f20a80b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34248/0 (socket says 172.21.15.98:34248) 2023-10-06T01:40:01.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 -- 172.21.15.98:0/3731990480 learned_addr learned my addr 172.21.15.98:0/3731990480 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 -- 172.21.15.98:0/1545827479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840cbc60 msgr2=0x7f2b841556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/1545827479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840cbc60 0x7f2b841556a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b840ab9a0 tx=0x7f2b740091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 -- 172.21.15.98:0/1545827479 shutdown_connections 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/1545827479 >> v2:172.21.15.98:3300/0 conn(0x7f2b8406e410 0x7f2b84069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/1545827479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840cbc60 0x7f2b841556a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 -- 172.21.15.98:0/1545827479 >> 172.21.15.98:0/1545827479 conn(0x7f2b840680c0 msgr2=0x7f2b840cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 -- 172.21.15.98:0/1545827479 shutdown_connections 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f5ca740f700 1 -- 172.21.15.98:0/1472202006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 msgr2=0x7f5ca0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f5ca740f700 1 --2- 172.21.15.98:0/1472202006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 0x7f5ca0069560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006fd0 tx=0x7f5c9c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 -- 172.21.15.98:0/1472202006 shutdown_connections 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 --2- 172.21.15.98:0/1472202006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca006e410 0x7f5ca0069560 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 --2- 172.21.15.98:0/1472202006 >> v2:172.21.15.98:3300/0 conn(0x7f5ca00c82a0 0x7f5ca0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 -- 172.21.15.98:0/1472202006 >> 172.21.15.98:0/1472202006 conn(0x7f5ca00680c0 msgr2=0x7f5ca00cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 -- 172.21.15.98:0/3731990480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a8162370 msgr2=0x7f20a80cb450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 --2- 172.21.15.98:0/3731990480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a8162370 0x7f20a80cb450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 -- 172.21.15.98:0/3731990480 >> v2:172.21.15.184:3300/0 conn(0x7f20a81617c0 msgr2=0x7f20a8161c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 --2- 172.21.15.98:0/3731990480 >> v2:172.21.15.184:3300/0 conn(0x7f20a81617c0 0x7f20a8161c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f20ad020700 1 -- 172.21.15.98:0/3731990480 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80cbd00 con 0x7f20a80b2390 2023-10-06T01:40:01.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bfeec700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3b8069c70 0x7fb3b80aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bfeec700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b80c7e80 con 0x7fb3b8161200 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bfeec700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b8161680 con 0x7fb3b8160e30 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bfeec700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80c8a20 con 0x7fb3b8069c70 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bd487700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb3b8161200 0x7fb3b8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36462/0 (socket says 172.21.15.98:36462) 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bd487700 1 -- 172.21.15.98:0/2889776101 learned_addr learned my addr 172.21.15.98:0/2889776101 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729122272 0 0) 0x7fb3b80c7e80 con 0x7fb3b8161200 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a4003570 con 0x7fb3b8161200 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1674353212 0 0) 0x7fb3a4003570 con 0x7fb3b8161200 2023-10-06T01:40:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 >> v2:172.21.15.184:3300/0 conn(0x7fb3b8069c70 msgr2=0x7fb3b80aeef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 --2- 172.21.15.98:0/2889776101 >> v2:172.21.15.184:3300/0 conn(0x7fb3b8069c70 0x7fb3b80aeef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 >> v1:172.21.15.184:6789/0 conn(0x7fb3b8160e30 legacy=0x7fb3b80c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80c7270 con 0x7fb3b8161200 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fb3b4001470 con 0x7fb3b8161200 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 >> v1:172.21.15.98:6789/0 conn(0x7fb3b8161200 legacy=0x7fb3b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4003ae0 0x7fb3a4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 0x7fb3a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a40053c0 0x7fb3a401ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80c8a20 con 0x7fb3a4004810 2023-10-06T01:40:01.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a401d2d0 con 0x7fb3a40053c0 2023-10-06T01:40:01.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a401d490 con 0x7fb3a4003ae0 2023-10-06T01:40:01.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f18f2d85700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18ec0b2390 0x7f18ec0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18f2d85700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec15e170 con 0x7f18ec1617c0 2023-10-06T01:40:01.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18f2d85700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0cbd00 con 0x7f18ec162370 2023-10-06T01:40:01.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18f2d85700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec15e340 con 0x7f18ec0b2390 2023-10-06T01:40:01.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 -- 172.21.15.98:0/1472202006 shutdown_connections 2023-10-06T01:40:01.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f5ca740f700 1 -- 172.21.15.98:0/1472202006 wait complete. 2023-10-06T01:40:01.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3be489700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a40053c0 0x7fb3a401ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 0x7fb3a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18f0b21700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18ec0b2390 0x7f18ec0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18ebfff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f18ec162370 0x7f18ec0cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35868/0 (socket says 172.21.15.98:35868) 2023-10-06T01:40:01.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18f0b21700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18ec0b2390 0x7f18ec0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34260/0 (socket says 172.21.15.98:34260) 2023-10-06T01:40:01.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18ebfff700 1 -- 172.21.15.98:0/3242956135 learned_addr learned my addr 172.21.15.98:0/3242956135 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:01.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fd60a5d6700 1 Processor -- start 2023-10-06T01:40:01.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f20ad020700 1 --2- 172.21.15.98:0/3731990480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80b2760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20a80cdea0 tx=0x7f20a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0ada897cfa585f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 -- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4003ae0 msgr2=0x7fb3a4003ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T01:40:01.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4003ae0 0x7fb3a4003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 -- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a40053c0 msgr2=0x7fb3a401ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a40053c0 0x7fb3a401ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3bd487700 1 -- 172.21.15.98:0/2889776101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80c7270 con 0x7fb3a4004810 2023-10-06T01:40:01.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7fb3be489700 1 -- 172.21.15.98:0/2889776101 reap_dead start 2023-10-06T01:40:01.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f7855ffb700 1 -- 172.21.15.98:0/302760013 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00c870 con 0x7f7858069c70 2023-10-06T01:40:01.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f7855ffb700 1 -- 172.21.15.98:0/302760013 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c00c9d0 con 0x7f7858069c70 2023-10-06T01:40:01.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f7855ffb700 1 -- 172.21.15.98:0/302760013 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00cd90 con 0x7f7858069c70 2023-10-06T01:40:01.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033437824 0 0) 0x7f18ec0cbd00 con 0x7f18ec162370 2023-10-06T01:40:01.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d4003570 con 0x7f18ec162370 2023-10-06T01:40:01.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7fb3bd487700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 0x7fb3a4004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c320 tx=0x7fb3b400c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37f5fe5f0e6ddbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411311508 0 0) 0x7f18d4003570 con 0x7f18ec162370 2023-10-06T01:40:01.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f18f0b21700 1 -- 172.21.15.98:0/3242956135 >> v1:172.21.15.184:6789/0 conn(0x7f18ec162370 legacy=0x7f18ec0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f18f0b21700 1 -- 172.21.15.98:0/3242956135 >> v1:172.21.15.98:6789/0 conn(0x7f18ec1617c0 legacy=0x7f18ec15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f18f0b21700 1 -- 172.21.15.98:0/3242956135 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0cbfe0 con 0x7f18ec0b2390 2023-10-06T01:40:01.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f18f0b21700 1 --2- 172.21.15.98:0/3242956135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec0b2390 0x7f18ec0b2760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009860 tx=0x7f18e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d746913756b8cdda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.054+0000 7f2b8abfd700 1 -- 172.21.15.98:0/1545827479 wait complete. 2023-10-06T01:40:01.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f2b8abfd700 1 Processor -- start 2023-10-06T01:40:01.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 -- 172.21.15.98:0/302760013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 msgr2=0x7f78580abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 --2- 172.21.15.98:0/302760013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580abcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f784c00aa70 tx=0x7f784c00a330 comp rx=0 tx=0).stop 2023-10-06T01:40:01.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f5ca740f700 1 Processor -- start 2023-10-06T01:40:01.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f209ffff700 1 -- 172.21.15.98:0/3731990480 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a000e070 con 0x7f20a80b2390 2023-10-06T01:40:01.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.055+0000 7f209ffff700 1 -- 172.21.15.98:0/3731990480 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a000ba50 con 0x7f20a80b2390 2023-10-06T01:40:01.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 -- 172.21.15.98:0/302760013 shutdown_connections 2023-10-06T01:40:01.086 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f209ffff700 1 -- 172.21.15.98:0/3731990480 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00092e0 con 0x7f20a80b2390 2023-10-06T01:40:01.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 --2- 172.21.15.98:0/302760013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580abcd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 --2- 172.21.15.98:0/302760013 >> v2:172.21.15.184:3300/0 conn(0x7f7858155860 0x7f78580ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 -- 172.21.15.98:0/302760013 >> 172.21.15.98:0/302760013 conn(0x7f78580680c0 msgr2=0x7f78580cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f2b8abfd700 1 -- start start 2023-10-06T01:40:01.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7fd60a5d6700 1 -- start start 2023-10-06T01:40:01.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 -- 172.21.15.98:0/302760013 shutdown_connections 2023-10-06T01:40:01.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f20af284700 1 -- 172.21.15.98:0/3731990480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 msgr2=0x7f20a80b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f20af284700 1 --2- 172.21.15.98:0/3731990480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80b2760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20a80cdea0 tx=0x7f20a0009840 comp rx=0 tx=0).stop 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f20af284700 1 -- 172.21.15.98:0/3731990480 shutdown_connections 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f20af284700 1 --2- 172.21.15.98:0/3731990480 >> v2:172.21.15.184:3300/0 conn(0x7f20a81617c0 0x7f20a8161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f20af284700 1 --2- 172.21.15.98:0/3731990480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80b2760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f20af284700 1 --2- 172.21.15.98:0/3731990480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a8162370 0x7f20a80cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f20af284700 1 -- 172.21.15.98:0/3731990480 >> 172.21.15.98:0/3731990480 conn(0x7f20a80ccf70 msgr2=0x7f20a80cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc001470 con 0x7f18ec0b2390 2023-10-06T01:40:01.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e0011420 con 0x7f18ec0b2390 2023-10-06T01:40:01.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f18eb7fe700 1 -- 172.21.15.98:0/3242956135 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e00117c0 con 0x7f18ec0b2390 2023-10-06T01:40:01.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b8406e410 0x7f2b841430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840c82a0 0x7f2b84176180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840c6070 con 0x7f2b840cbc60 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840c06e0 con 0x7f2b840c82a0 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8abfd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840c2ab0 con 0x7f2b8406e410 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8919a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8919a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34266/0 (socket says 172.21.15.98:34266) 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f2b8919a700 1 -- 172.21.15.98:0/34515256 learned_addr learned my addr 172.21.15.98:0/34515256 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.057+0000 7f5ca740f700 1 -- start start 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd60a5d6700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd60406dba0 0x7fd6040c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd60a5d6700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60406ac80 con 0x7fd60406df70 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd60a5d6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60406b820 con 0x7fd604069b20 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd60a5d6700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60406a070 con 0x7fd60406dba0 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f20af284700 1 -- 172.21.15.98:0/3731990480 shutdown_connections 2023-10-06T01:40:01.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f20af284700 1 -- 172.21.15.98:0/3731990480 wait complete. 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 -- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b8406e410 msgr2=0x7f2b841430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b8406e410 0x7f2b841430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 -- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840c82a0 msgr2=0x7f2b84176180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840c82a0 0x7f2b84176180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 -- 172.21.15.98:0/34515256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74009420 con 0x7f2b840cbc60 2023-10-06T01:40:01.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f18f2d85700 1 -- 172.21.15.98:0/3242956135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec0b2390 msgr2=0x7f18ec0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f18f2d85700 1 --2- 172.21.15.98:0/3242956135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec0b2390 0x7f18ec0b2760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009860 tx=0x7f18e00091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f7a53405700 1 Processor -- start 2023-10-06T01:40:01.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd60406dba0 0x7fd6040c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd60406dba0 0x7fd6040c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:49958/0 (socket says 172.21.15.98:49958) 2023-10-06T01:40:01.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 -- 172.21.15.98:0/2369748940 learned_addr learned my addr 172.21.15.98:0/2369748940 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f18f2d85700 1 -- 172.21.15.98:0/3242956135 shutdown_connections 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f18f2d85700 1 --2- 172.21.15.98:0/3242956135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec0b2390 0x7f18ec0b2760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f18f2d85700 1 -- 172.21.15.98:0/3242956135 >> 172.21.15.98:0/3242956135 conn(0x7f18ec0ccf70 msgr2=0x7f18ec0cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400ccc0 con 0x7fb3a4004810 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b400ce20 con 0x7fb3a4004810 2023-10-06T01:40:01.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fb3bcc86700 1 -- 172.21.15.98:0/2889776101 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4005b50 con 0x7fb3a4004810 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca006e410 0x7f5ca014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 0x7f5ca00c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 0x7f5ca00c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c0530 con 0x7f5ca00cbc60 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c1340 con 0x7f5ca006e410 2023-10-06T01:40:01.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca740f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c3f10 con 0x7f5ca00c82a0 2023-10-06T01:40:01.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd6037fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd604069b20 0x7fd60406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35870/0 (socket says 172.21.15.98:35870) 2023-10-06T01:40:01.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.056+0000 7f785da2f700 1 -- 172.21.15.98:0/302760013 wait complete. 2023-10-06T01:40:01.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca49aa700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 0x7f5ca00c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f785da2f700 1 Processor -- start 2023-10-06T01:40:01.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 -- 172.21.15.98:0/2369748940 >> v1:172.21.15.184:6789/0 conn(0x7fd604069b20 legacy=0x7fd60406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f2b8919a700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009e20 tx=0x7f2b7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 -- 172.21.15.98:0/2369748940 >> v1:172.21.15.98:6789/0 conn(0x7fd60406df70 legacy=0x7fd6040ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7fd603fff700 1 -- 172.21.15.98:0/2369748940 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60406a350 con 0x7fd60406dba0 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca49aa700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 0x7f5ca00c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:58930/0 (socket says 172.21.15.98:58930) 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca49aa700 1 -- 172.21.15.98:0/551763833 learned_addr learned my addr 172.21.15.98:0/551763833 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca51ab700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca006e410 0x7f5ca014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 0x7f5ca00c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fd603fff700 1 --2- 172.21.15.98:0/2369748940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60406dba0 0x7fd6040c7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800bc40 tx=0x7fd5f800b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57234a443f35e548 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c009320 con 0x7f2b840cbc60 2023-10-06T01:40:01.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c00f040 con 0x7f2b840cbc60 2023-10-06T01:40:01.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00cc40 con 0x7f2b840cbc60 2023-10-06T01:40:01.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fd602ffd700 1 -- 172.21.15.98:0/2369748940 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800dac0 con 0x7fd60406dba0 2023-10-06T01:40:01.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fd602ffd700 1 -- 172.21.15.98:0/2369748940 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f800dc20 con 0x7fd60406dba0 2023-10-06T01:40:01.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fd602ffd700 1 -- 172.21.15.98:0/2369748940 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8017410 con 0x7fd60406dba0 2023-10-06T01:40:01.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f5ca59ac700 1 -- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 msgr2=0x7f5ca00c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 0x7f5ca00c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f5ca59ac700 1 -- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca006e410 msgr2=0x7f5ca014aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca006e410 0x7f5ca014aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f5ca59ac700 1 -- 172.21.15.98:0/551763833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009420 con 0x7f5ca00cbc60 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f7a53405700 1 -- start start 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f7a53405700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7a4c068d10 0x7f7a4c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f7a53405700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0c9750 con 0x7f7a4c0b2460 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f7a53405700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0b2910 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f7a53405700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8ef0 con 0x7f7a4c068d10 2023-10-06T01:40:01.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fb3bfeec700 1 -- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 msgr2=0x7fb3a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 0x7fb3a4004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c320 tx=0x7fb3b400c630 comp rx=0 tx=0).stop 2023-10-06T01:40:01.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5ca59ac700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 0x7f5ca00c3130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c014000 tx=0x7f5c9c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f20af284700 1 Processor -- start 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.058+0000 7f785da2f700 1 -- start start 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 0x7f7858175dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 0x7f7858176300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785817ac20 con 0x7f78580c82a0 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858176840 con 0x7f7858155860 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.059+0000 7f785da2f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785817a050 con 0x7f7858069c70 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78567fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 0x7f7858175dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78567fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 0x7f7858175dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34292/0 (socket says 172.21.15.98:34292) 2023-10-06T01:40:01.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78567fc700 1 -- 172.21.15.98:0/1416108288 learned_addr learned my addr 172.21.15.98:0/1416108288 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7856ffd700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 0x7f7858176300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f20af284700 1 -- start start 2023-10-06T01:40:01.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f18f2d85700 1 -- 172.21.15.98:0/3242956135 shutdown_connections 2023-10-06T01:40:01.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f18f2d85700 1 -- 172.21.15.98:0/3242956135 wait complete. 2023-10-06T01:40:01.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a511a1700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f7a4c15d8f0 0x7f7a4c0b1d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:35874/0 (socket says 172.21.15.98:35874) 2023-10-06T01:40:01.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a511a1700 1 -- 172.21.15.98:0/449251565 learned_addr learned my addr 172.21.15.98:0/449251565 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:01.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b840c09a0 con 0x7f2b840cbc60 2023-10-06T01:40:01.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b84143680 con 0x7f2b840cbc60 2023-10-06T01:40:01.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fb3bfeec700 1 -- 172.21.15.98:0/2889776101 shutdown_connections 2023-10-06T01:40:01.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/2889776101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3a4004810 0x7fb3a4004c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fb3bfeec700 1 -- 172.21.15.98:0/2889776101 >> 172.21.15.98:0/2889776101 conn(0x7fb3b80680c0 msgr2=0x7fb3b80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fb3bfeec700 1 -- 172.21.15.98:0/2889776101 shutdown_connections 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 -- 172.21.15.98:0/2369748940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60406dba0 msgr2=0x7fd6040c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/2369748940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60406dba0 0x7fd6040c7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800bc40 tx=0x7fd5f800b4e0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 -- 172.21.15.98:0/2369748940 shutdown_connections 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/2369748940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60406dba0 0x7fd6040c7bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 -- 172.21.15.98:0/2369748940 >> 172.21.15.98:0/2369748940 conn(0x7fd6040cc730 msgr2=0x7fd6040ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 -- 172.21.15.98:0/2369748940 shutdown_connections 2023-10-06T01:40:01.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c005c60 con 0x7f5ca00cbc60 2023-10-06T01:40:01.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0062b0 con 0x7f5ca00cbc60 2023-10-06T01:40:01.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c011b60 con 0x7f5ca00cbc60 2023-10-06T01:40:01.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca00c0730 con 0x7f5ca00cbc60 2023-10-06T01:40:01.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca00c0cf0 con 0x7f5ca00cbc60 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261661763 0 0) 0x7f7a4c0b2910 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c003570 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378944862 0 0) 0x7f7a4c0c9750 con 0x7f7a4c0b2460 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c0b2910 con 0x7f7a4c0b2460 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2207364640 0 0) 0x7f7a3c003570 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 >> v2:172.21.15.184:3300/0 conn(0x7f7a4c068d10 msgr2=0x7f7a4c15dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 --2- 172.21.15.98:0/449251565 >> v2:172.21.15.184:3300/0 conn(0x7f7a4c068d10 0x7f7a4c15dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 >> v1:172.21.15.98:6789/0 conn(0x7f7a4c0b2460 legacy=0x7f7a4c068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0c9100 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f7a40005260 con 0x7f7a4c15d8f0 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 >> v1:172.21.15.184:6789/0 conn(0x7f7a4c15d8f0 legacy=0x7f7a4c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a3c003bf0 0x7f7a3c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a3c005310 0x7f7a3c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8ef0 con 0x7f7a3c005310 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c00c230 con 0x7f7a3c004810 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c00c3f0 con 0x7f7a3c003bf0 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a509a0700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a511a1700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a3c003bf0 0x7f7a3c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a509a0700 1 -- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a3c003bf0 msgr2=0x7f7a3c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f7a509a0700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a3c003bf0 0x7f7a3c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a509a0700 1 -- 172.21.15.98:0/449251565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a3c005310 msgr2=0x7f7a3c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a509a0700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a3c005310 0x7f7a3c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a509a0700 1 -- 172.21.15.98:0/449251565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0c9100 con 0x7f7a3c004810 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a519a2700 1 -- 172.21.15.98:0/449251565 reap_dead start 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b840c0510 con 0x7f2b840cbc60 2023-10-06T01:40:01.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7fb3bfeec700 1 -- 172.21.15.98:0/2889776101 wait complete. 2023-10-06T01:40:01.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7fd60a5d6700 1 -- 172.21.15.98:0/2369748940 wait complete. 2023-10-06T01:40:01.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a81617c0 0x7f20a8157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f18f2d85700 1 Processor -- start 2023-10-06T01:40:01.114 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a509a0700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a3c004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001a40 tx=0x7f7a38007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c88677d4c8a9d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80ca400 con 0x7f20a80b2390 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c97f0 con 0x7f20a8162370 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20af284700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c9ad0 con 0x7f20a81617c0 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f20ad821700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:50012/0 (socket says 172.21.15.98:50012) 2023-10-06T01:40:01.115 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 -- 172.21.15.98:0/3345749606 learned_addr learned my addr 172.21.15.98:0/3345749606 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 -- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 msgr2=0x7f78580cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 -- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 msgr2=0x7f7858175dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 0x7f7858175dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 -- 172.21.15.98:0/1416108288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c00a5b0 con 0x7f7858155860 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.060+0000 7f78577fe700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 0x7f7858176300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f784c016000 tx=0x7f784c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00c870 con 0x7f7858155860 2023-10-06T01:40:01.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c01c040 con 0x7f7858155860 2023-10-06T01:40:01.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c016870 con 0x7f7858155860 2023-10-06T01:40:01.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7fd60a5d6700 1 Processor -- start 2023-10-06T01:40:01.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 -- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a81617c0 msgr2=0x7f20a8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a81617c0 0x7f20a8157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 -- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 msgr2=0x7f20a80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785817a2f0 con 0x7f7858155860 2023-10-06T01:40:01.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 -- 172.21.15.98:0/3345749606 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009ae0 con 0x7f20a8162370 2023-10-06T01:40:01.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f18f2d85700 1 -- start start 2023-10-06T01:40:01.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785817a8b0 con 0x7f7858155860 2023-10-06T01:40:01.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800c5d0 con 0x7f7a3c004810 2023-10-06T01:40:01.120 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800c730 con 0x7f7a3c004810 2023-10-06T01:40:01.120 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f7a4bfff700 1 -- 172.21.15.98:0/449251565 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cad0 con 0x7f7a3c004810 2023-10-06T01:40:01.120 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.062+0000 7f20ad821700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20a400b5e0 tx=0x7f20a4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.120 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.061+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800052b0 con 0x7f5ca00cbc60 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 -- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 msgr2=0x7f7a3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a3c004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001a40 tx=0x7f7a38007d10 comp rx=0 tx=0).stop 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7fb3bfeec700 1 Processor -- start 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 -- 172.21.15.98:0/449251565 shutdown_connections 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 --2- 172.21.15.98:0/449251565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a3c004c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 -- 172.21.15.98:0/449251565 >> 172.21.15.98:0/449251565 conn(0x7f7a4c06de80 msgr2=0x7f7a4c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7fb3bfeec700 1 -- start start 2023-10-06T01:40:01.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785817add0 con 0x7f7858155860 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 0x7f18ec0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec162370 0x7f18ec0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0c6510 con 0x7f18ec162370 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0c76b0 con 0x7f18ec1617c0 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f18f2d85700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0c7950 con 0x7f18ec0b2390 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 0x7f18ec0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:50026/0 (socket says 172.21.15.98:50026) 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18ebfff700 1 -- 172.21.15.98:0/1924558253 learned_addr learned my addr 172.21.15.98:0/1924558253 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f1322700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec162370 0x7f18ec0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 -- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 msgr2=0x7f18ec16e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 -- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec162370 msgr2=0x7f18ec0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec162370 0x7f18ec0c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 -- 172.21.15.98:0/1924558253 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009420 con 0x7f18ec0b2390 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18ebfff700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:01.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c9c023020 con 0x7f5ca00cbc60 2023-10-06T01:40:01.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 -- 172.21.15.98:0/449251565 shutdown_connections 2023-10-06T01:40:01.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7f7a53405700 1 -- 172.21.15.98:0/449251565 wait complete. 2023-10-06T01:40:01.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18f0b21700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 0x7f18ec0cb3a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c74a0 tx=0x7f18e0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f5c957fa700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 0x7f5c840694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5c9c086790 con 0x7f5ca00cbc60 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f5ca51ab700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 0x7f5c840694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f510 con 0x7f20a8162370 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a400f690 con 0x7f20a8162370 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400fa50 con 0x7f20a8162370 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a80c9db0 con 0x7f20a8162370 2023-10-06T01:40:01.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a80cb260 con 0x7f20a8162370 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e001caf0 con 0x7f18ec0b2390 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e001cc50 con 0x7f18ec0b2390 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e001a450 con 0x7f18ec0b2390 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec0c7c30 con 0x7f18ec0b2390 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec0c6200 con 0x7f18ec0b2390 2023-10-06T01:40:01.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f7a53405700 1 Processor -- start 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b7c014650 con 0x7f2b840cbc60 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f2b80ff9700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 0x7f2b70069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2b7c085e20 con 0x7f2b840cbc60 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 0x7fb3b80aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3b80af410 0x7fb3b80c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8156cc0 con 0x7fb3b80c4640 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80c62b0 con 0x7fb3b80af410 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fb3bfeec700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80af880 con 0x7fb3a4004810 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3be489700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3be489700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34338/0 (socket says 172.21.15.98:34338) 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3be489700 1 -- 172.21.15.98:0/3437683059 learned_addr learned my addr 172.21.15.98:0/3437683059 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 0x7fb3b80aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.063+0000 7fd60a5d6700 1 -- start start 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 0x7fd6040bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60406df70 0x7fd6040c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604158e30 con 0x7fd60406dba0 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040bb520 con 0x7fd604069b20 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.064+0000 7fd60a5d6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040bc410 con 0x7fd60406df70 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fd603fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fd603fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:50030/0 (socket says 172.21.15.98:50030) 2023-10-06T01:40:01.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fd603fff700 1 -- 172.21.15.98:0/87619933 learned_addr learned my addr 172.21.15.98:0/87619933 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fd603fff700 1 -- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60406df70 msgr2=0x7fd6040c3af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fd603fff700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60406df70 0x7fd6040c3af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fd603fff700 1 -- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 msgr2=0x7fd6040bbde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec0c8150 con 0x7f18ec0b2390 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bd487700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3b80af410 0x7fb3b80c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 -- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3b80af410 msgr2=0x7fb3b80c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3b80af410 0x7fb3b80c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 -- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 msgr2=0x7fb3b80c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7fb3bdc88700 1 -- 172.21.15.98:0/3437683059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80bfd60 con 0x7fb3a4004810 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fb3bdc88700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 0x7fb3b80aeed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b520 tx=0x7fb3a8009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800f510 con 0x7fb3a4004810 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a800f690 con 0x7fb3a4004810 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800fa50 con 0x7fb3a4004810 2023-10-06T01:40:01.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7fb3be489700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.065+0000 7f7a53405700 1 -- start start 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4c0a7ed0 0x7f7a4c0a8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a4c0a54d0 0x7f7a4c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c172650 con 0x7f7a4c0a54d0 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0a5e60 con 0x7f7a3c004810 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a53405700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c172370 con 0x7f7a4c0a7ed0 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f7a511a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:50048/0 (socket says 172.21.15.98:50048) 2023-10-06T01:40:01.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 -- 172.21.15.98:0/4166061167 learned_addr learned my addr 172.21.15.98:0/4166061167 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:01.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f784c016c20 con 0x7f7858155860 2023-10-06T01:40:01.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f783effd700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 0x7f7844069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f784c013070 con 0x7f7858155860 2023-10-06T01:40:01.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7856ffd700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 0x7f7844069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 -- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4c0a7ed0 msgr2=0x7f7a4c0a8340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4c0a7ed0 0x7f7a4c0a8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 -- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a4c0a54d0 msgr2=0x7f7a4c0a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a4c0a54d0 0x7f7a4c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 -- 172.21.15.98:0/4166061167 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0a73f0 con 0x7f7a3c004810 2023-10-06T01:40:01.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b8160a30 con 0x7fb3a4004810 2023-10-06T01:40:01.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fd6037fe700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 0x7fd6040bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fd603fff700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 0x7fd6040bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7a511a1700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ef60 tx=0x7f7a40009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b8156700 con 0x7fb3a4004810 2023-10-06T01:40:01.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fd603fff700 1 -- 172.21.15.98:0/87619933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f800b780 con 0x7fd604069b20 2023-10-06T01:40:01.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fd6037fe700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 0x7fd6040bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:01.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7fd603fff700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd604069720 tx=0x7fd5f8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f7856ffd700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 0x7f7844069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7848006fd0 tx=0x7f7848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18e0044bd0 con 0x7f18ec0b2390 2023-10-06T01:40:01.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.068+0000 7f18e8ff9700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.068+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f18e0086420 con 0x7f18ec0b2390 2023-10-06T01:40:01.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.068+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b80bfb30 con 0x7fb3a4004810 2023-10-06T01:40:01.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.070+0000 7f18ebfff700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.070+0000 7f5ca51ab700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 0x7f5c840694b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009db0 tx=0x7f5c900091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.070+0000 7f18ebfff700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 0x7f18d80693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18ec06f850 tx=0x7f18dc0019e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.066+0000 7f2b88999700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 0x7f2b70069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.067+0000 7f2b88999700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 0x7f2b70069460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b78001720 tx=0x7f2b78007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20a401a400 con 0x7f20a8162370 2023-10-06T01:40:01.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a8064ad0 con 0x7f20a8162370 2023-10-06T01:40:01.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000ccf0 con 0x7f7a3c004810 2023-10-06T01:40:01.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8025070 con 0x7fd604069b20 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f80165d0 con 0x7fd604069b20 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8009040 con 0x7fd604069b20 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6040bc6f0 con 0x7fd604069b20 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6040bccb0 con 0x7fd604069b20 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000ce90 con 0x7f7a3c004810 2023-10-06T01:40:01.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c0a76d0 con 0x7f7a3c004810 2023-10-06T01:40:01.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f785817add0 con 0x7f7858155860 2023-10-06T01:40:01.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c06b710 con 0x7f7a3c004810 2023-10-06T01:40:01.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40005be0 con 0x7f7a3c004810 2023-10-06T01:40:01.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.072+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18e00532c0 con 0x7f18ec0b2390 2023-10-06T01:40:01.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f209d7fa700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 0x7f208c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f20a40844f0 con 0x7f20a8162370 2023-10-06T01:40:01.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.071+0000 7f20ad020700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 0x7f208c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.072+0000 7f20ad020700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 0x7f208c069530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f20a80cdea0 tx=0x7f20a0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.072+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd604064ad0 con 0x7fd604069b20 2023-10-06T01:40:01.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.072+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c0052b0 con 0x7f7a3c004810 2023-10-06T01:40:01.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.073+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b7c052c20 con 0x7f2b840cbc60 2023-10-06T01:40:01.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.073+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3a8024430 con 0x7fb3a4004810 2023-10-06T01:40:01.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.073+0000 7fb3adffb700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.073+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb3a8084b00 con 0x7fb3a4004810 2023-10-06T01:40:01.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.074+0000 7fb3bd487700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.074+0000 7fb3bd487700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 0x7fb39c0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4006fd0 tx=0x7fb3b4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.147 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.076+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20a4054e00 con 0x7f20a8162370 2023-10-06T01:40:01.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.077+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c9c053710 con 0x7f5ca00cbc60 2023-10-06T01:40:01.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.077+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a40007070 con 0x7f7a3c004810 2023-10-06T01:40:01.148 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7f7a497fa700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7a40085b20 con 0x7f7a3c004810 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7f7a509a0700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.077+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5f8015070 con 0x7fd604069b20 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7fd5ebfff700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 0x7fd5f00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd5f8086d80 con 0x7fd604069b20 2023-10-06T01:40:01.149 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7fd6037fe700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 0x7fd5f00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:01.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.079+0000 7f7a509a0700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001a40 tx=0x7f7a38015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.079+0000 7fd6037fe700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 0x7fd5f00694b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40051a0 tx=0x7fd5f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:01.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.081+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a40052920 con 0x7f7a3c004810 2023-10-06T01:40:01.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.085+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5f8053c80 con 0x7fd604069b20 2023-10-06T01:40:01.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.078+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3a8051a80 con 0x7fb3a4004810 2023-10-06T01:40:01.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.452+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f785817add0 con 0x7f7858155860 2023-10-06T01:40:01.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.454+0000 7f783effd700 1 -- 172.21.15.98:0/1416108288 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f785817add0 con 0x7f7858155860 2023-10-06T01:40:01.458 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2023-10-06T01:40:01.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.454+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f18ec0c8150 con 0x7f18ec0b2390 2023-10-06T01:40:01.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.456+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2b840c0b90 con 0x7f2b840cbc60 2023-10-06T01:40:01.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.457+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 msgr2=0x7f7844069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.457+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 0x7f7844069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7848006fd0 tx=0x7f7848008040 comp rx=0 tx=0).stop 2023-10-06T01:40:01.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.457+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 msgr2=0x7f7858176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.457+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 0x7f7858176300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f784c016000 tx=0x7f784c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.458+0000 7f18e8ff9700 1 -- 172.21.15.98:0/1924558253 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18e00568e0 con 0x7f18ec0b2390 2023-10-06T01:40:01.462 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:01.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 shutdown_connections 2023-10-06T01:40:01.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7844067080 0x7f7844069530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7858155860 0x7f7858176300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7858069c70 0x7f78580cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 --2- 172.21.15.98:0/1416108288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78580c82a0 0x7f7858175dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 >> 172.21.15.98:0/1416108288 conn(0x7f78580680c0 msgr2=0x7f7858154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.460+0000 7f2b80ff9700 1 -- 172.21.15.98:0/34515256 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b7c056240 con 0x7f2b840cbc60 2023-10-06T01:40:01.465 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:01.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.462+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 shutdown_connections 2023-10-06T01:40:01.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.462+0000 7f785da2f700 1 -- 172.21.15.98:0/1416108288 wait complete. 2023-10-06T01:40:01.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 msgr2=0x7f18d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 0x7f18d80693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18ec06f850 tx=0x7f18dc0019e0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 msgr2=0x7f18ec0cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 0x7f18ec0cb3a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c74a0 tx=0x7f18e0016000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 msgr2=0x7f2b70069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 0x7f2b70069460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b78001720 tx=0x7f2b78007ab0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 msgr2=0x7f2b840c0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009e20 tx=0x7f2b7c00b040 comp rx=0 tx=0).stop 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 shutdown_connections 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2b70066fb0 0x7f2b70069460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b840cbc60 0x7f2b840c0ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b8406e410 0x7f2b841430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 --2- 172.21.15.98:0/34515256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b840c82a0 0x7f2b84176180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 >> 172.21.15.98:0/34515256 conn(0x7f2b840680c0 msgr2=0x7f2b840c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 shutdown_connections 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18ec162370 0x7f18ec0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18ec0b2390 0x7f18ec0cb3a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 --2- 172.21.15.98:0/1924558253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18ec1617c0 0x7f18ec16e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 >> 172.21.15.98:0/1924558253 conn(0x7f18ec0ccf70 msgr2=0x7f18ec15f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 shutdown_connections 2023-10-06T01:40:01.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f2b8abfd700 1 -- 172.21.15.98:0/34515256 wait complete. 2023-10-06T01:40:01.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb3b80bfb30 con 0x7fb3a4004810 2023-10-06T01:40:01.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7fb3adffb700 1 -- 172.21.15.98:0/3437683059 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3b80bfb30 con 0x7fb3a4004810 2023-10-06T01:40:01.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 shutdown_connections 2023-10-06T01:40:01.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.464+0000 7f18f2d85700 1 -- 172.21.15.98:0/1924558253 wait complete. 2023-10-06T01:40:01.470 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2023-10-06T01:40:01.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.465+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f20a80ca540 con 0x7f20a8162370 2023-10-06T01:40:01.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.465+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c80005830 con 0x7f5ca00cbc60 2023-10-06T01:40:01.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.465+0000 7f209d7fa700 1 -- 172.21.15.98:0/3345749606 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20a4054990 con 0x7f20a8162370 2023-10-06T01:40:01.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.465+0000 7f5c957fa700 1 -- 172.21.15.98:0/551763833 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c9c02b090 con 0x7f5ca00cbc60 2023-10-06T01:40:01.472 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:01.472 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:01.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 msgr2=0x7fb39c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 0x7fb39c0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4006fd0 tx=0x7fb3b4015000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 msgr2=0x7fb3b80aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 0x7fb3b80aeed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b520 tx=0x7fb3a8009060 comp rx=0 tx=0).stop 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 msgr2=0x7f5c840694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 0x7f5c840694b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009db0 tx=0x7f5c900091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 msgr2=0x7f5ca00c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 0x7f5ca00c3130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c014000 tx=0x7f5c9c015000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 msgr2=0x7f208c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 0x7f208c069530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f20a80cdea0 tx=0x7f20a0018000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 msgr2=0x7f20a8158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20a400b5e0 tx=0x7f20a4009120 comp rx=0 tx=0).stop 2023-10-06T01:40:01.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 shutdown_connections 2023-10-06T01:40:01.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80c4640 0x7fb3b80c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3a4004810 0x7fb3b80aeed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 --2- 172.21.15.98:0/3437683059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3b80af410 0x7fb3b80c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 shutdown_connections 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f208c067080 0x7f208c069530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20a8162370 0x7f20a8158310 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20a80b2390 0x7f20a80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 --2- 172.21.15.98:0/3345749606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20a81617c0 0x7f20a8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 >> 172.21.15.98:0/3437683059 conn(0x7fb3b80680c0 msgr2=0x7fb3b8157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 >> 172.21.15.98:0/3345749606 conn(0x7f20a80ccf70 msgr2=0x7f20a80b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 shutdown_connections 2023-10-06T01:40:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 shutdown_connections 2023-10-06T01:40:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7fb3bfeec700 1 -- 172.21.15.98:0/3437683059 wait complete. 2023-10-06T01:40:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.469+0000 7f20af284700 1 -- 172.21.15.98:0/3345749606 wait complete. 2023-10-06T01:40:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.472+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7a2c005830 con 0x7f7a3c004810 2023-10-06T01:40:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.472+0000 7f7a497fa700 1 -- 172.21.15.98:0/4166061167 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a40055f40 con 0x7f7a3c004810 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 shutdown_connections 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5c84067000 0x7f5c840694b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca00cbc60 0x7f5ca00c3130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca006e410 0x7f5ca014aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 --2- 172.21.15.98:0/551763833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca00c82a0 0x7f5ca00c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 >> 172.21.15.98:0/551763833 conn(0x7f5ca00680c0 msgr2=0x7f5ca0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 shutdown_connections 2023-10-06T01:40:01.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 msgr2=0x7f7a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001a40 tx=0x7f7a38015000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 msgr2=0x7f7a4c06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ef60 tx=0x7f7a40009040 comp rx=0 tx=0).stop 2023-10-06T01:40:01.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f5ca740f700 1 -- 172.21.15.98:0/551763833 wait complete. 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 shutdown_connections 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a4c0a54d0 0x7f7a4c0a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a3c004810 0x7f7a4c06b1d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 --2- 172.21.15.98:0/4166061167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4c0a7ed0 0x7f7a4c0a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.475+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 >> 172.21.15.98:0/4166061167 conn(0x7f7a4c06de80 msgr2=0x7f7a4c15c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.476+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 shutdown_connections 2023-10-06T01:40:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.476+0000 7f7a53405700 1 -- 172.21.15.98:0/4166061167 wait complete. 2023-10-06T01:40:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.478+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd604064ad0 con 0x7fd604069b20 2023-10-06T01:40:01.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-10-06T01:40:01.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-10-06T01:40:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.490+0000 7fd5ebfff700 1 -- 172.21.15.98:0/87619933 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5f80572a0 con 0x7fd604069b20 2023-10-06T01:40:01.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-06T01:40:01.493 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 msgr2=0x7fd5f00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 0x7fd5f00694b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40051a0 tx=0x7fd5f4009040 comp rx=0 tx=0).stop 2023-10-06T01:40:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 msgr2=0x7fd6040bb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd604069720 tx=0x7fd5f8019000 comp rx=0 tx=0).stop 2023-10-06T01:40:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 shutdown_connections 2023-10-06T01:40:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd5f0067000 0x7fd5f00694b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60406df70 0x7fd6040c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604069b20 0x7fd6040bb8a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 --2- 172.21.15.98:0/87619933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd60406dba0 0x7fd6040bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.495+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 >> 172.21.15.98:0/87619933 conn(0x7fd6040cc730 msgr2=0x7fd6040ca260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.496+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 shutdown_connections 2023-10-06T01:40:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:01.496+0000 7fd60a5d6700 1 -- 172.21.15.98:0/87619933 wait complete. 2023-10-06T01:40:01.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-10-06T01:40:01.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-10-06T01:40:01.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-10-06T01:40:01.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-10-06T01:40:01.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-10-06T01:40:02.492 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T01:40:02.493 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T01:40:02.493 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T01:40:02.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T01:40:02.503 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T01:40:02.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T01:40:02.506 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T01:40:02.512 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T01:40:02.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fe6fac07700 1 Processor -- start 2023-10-06T01:40:02.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.685+0000 7fe6fac07700 1 -- start start 2023-10-06T01:40:02.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.683+0000 7fc35e7cd700 1 Processor -- start 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.683+0000 7fc35e7cd700 1 -- start start 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35e7cd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc35806d550 0x7fc35806d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35e7cd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35e7cd700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3580c87c0 con 0x7fc3580c8320 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35e7cd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580c8a20 con 0x7fc35806d550 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35e7cd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c7080 con 0x7fc3580878f0 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc3577fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc3580c8320 0x7fc358069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43070/0 (socket says 172.21.15.98:43070) 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc357fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc35806d550 0x7fc35806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc3577fe700 1 -- 172.21.15.98:0/1689655008 learned_addr learned my addr 172.21.15.98:0/1689655008 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.684+0000 7fc35cd6a700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.685+0000 7fc356ffd700 1 -- 172.21.15.98:0/1689655008 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106796304 0 0) 0x7fc3580c87c0 con 0x7fc3580c8320 2023-10-06T01:40:02.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc356ffd700 1 -- 172.21.15.98:0/1689655008 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc340003570 con 0x7fc3580c8320 2023-10-06T01:40:02.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fb735810700 1 Processor -- start 2023-10-06T01:40:02.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fe6fac07700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6f4067840 0x7fe6f40c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fe6fac07700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f406bfe0 con 0x7fe6f40caa30 2023-10-06T01:40:02.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fe6fac07700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f4162080 con 0x7fe6f40c7fa0 2023-10-06T01:40:02.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fe6fac07700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f4162360 con 0x7fe6f4067840 2023-10-06T01:40:02.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f91a4700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fe6f40c7fa0 0x7fe6f4160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:39234/0 (socket says 172.21.15.98:39234) 2023-10-06T01:40:02.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f91a4700 1 -- 172.21.15.98:0/985167625 learned_addr learned my addr 172.21.15.98:0/985167625 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f3fff700 1 --1- 172.21.15.98:0/985167625 >> v1:172.21.15.98:6789/0 conn(0x7fe6f40caa30 0x7fe6f406c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43082/0 (socket says 172.21.15.98:43082) 2023-10-06T01:40:02.690 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fb735810700 1 -- start start 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc357fff700 1 -- 172.21.15.98:0/1689655008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 msgr2=0x7fc358069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc357fff700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc357fff700 1 -- 172.21.15.98:0/1689655008 >> v1:172.21.15.98:6789/0 conn(0x7fc3580c8320 legacy=0x7fc358069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc357fff700 1 -- 172.21.15.98:0/1689655008 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580aff00 con 0x7fc35806d550 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.686+0000 7fc357fff700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc35806d920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3480095b0 tx=0x7fc348009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7d0506c9b87723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fc356ffd700 1 -- 172.21.15.98:0/1689655008 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800c040 con 0x7fc35806d550 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fc356ffd700 1 -- 172.21.15.98:0/1689655008 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3480092e0 con 0x7fc35806d550 2023-10-06T01:40:02.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fc356ffd700 1 -- 172.21.15.98:0/1689655008 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348013a00 con 0x7fc35806d550 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f89a3700 1 --2- 172.21.15.98:0/985167625 >> v2:172.21.15.98:3300/0 conn(0x7fe6f4067840 0x7fe6f40c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404991 0 0) 0x7fe6f406bfe0 con 0x7fe6f40caa30 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6dc003570 con 0x7fe6f40caa30 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f89a3700 1 -- 172.21.15.98:0/985167625 >> v1:172.21.15.184:6789/0 conn(0x7fe6f40c7fa0 legacy=0x7fe6f4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059808009 0 0) 0x7fe6dc003570 con 0x7fe6f40caa30 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f89a3700 1 -- 172.21.15.98:0/985167625 >> v1:172.21.15.98:6789/0 conn(0x7fe6f40caa30 legacy=0x7fe6f406c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.692 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f89a3700 1 -- 172.21.15.98:0/985167625 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4162640 con 0x7fe6f4067840 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f89a3700 1 --2- 172.21.15.98:0/985167625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f4067840 0x7fe6f40c7a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80098e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183bbdf213781876 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec001470 con 0x7fe6f4067840 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8011420 con 0x7fe6f4067840 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fe6f37fe700 1 -- 172.21.15.98:0/985167625 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80117c0 con 0x7fe6f4067840 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fc35e7cd700 1 -- 172.21.15.98:0/1689655008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 msgr2=0x7fc35806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc35806d920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3480095b0 tx=0x7fc348009f00 comp rx=0 tx=0).stop 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fb735810700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb7300c82a0 0x7fb730069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.687+0000 7fb735810700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb73006e410 0x7fb730069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb735810700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300c7e10 con 0x7fb730155860 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb735810700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300acce0 con 0x7fb7300c82a0 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb735810700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300c7200 con 0x7fb73006e410 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb72effd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb730155860 0x7fb7300ae4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43090/0 (socket says 172.21.15.98:43090) 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb72effd700 1 -- 172.21.15.98:0/533614625 learned_addr learned my addr 172.21.15.98:0/533614625 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950189212 0 0) 0x7fb7300c7e10 con 0x7fb730155860 2023-10-06T01:40:02.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.688+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb718003570 con 0x7fb730155860 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/533614625 >> v2:172.21.15.98:3300/0 conn(0x7fb7300c82a0 0x7fb730069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 262205249 0 0) 0x7fb718003570 con 0x7fb730155860 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 >> v2:172.21.15.184:3300/0 conn(0x7fb73006e410 msgr2=0x7fb730069aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 --2- 172.21.15.98:0/533614625 >> v2:172.21.15.184:3300/0 conn(0x7fb73006e410 0x7fb730069aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 >> v2:172.21.15.98:3300/0 conn(0x7fb7300c82a0 msgr2=0x7fb730069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 --2- 172.21.15.98:0/533614625 >> v2:172.21.15.98:3300/0 conn(0x7fb7300c82a0 0x7fb730069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300c74e0 con 0x7fb730155860 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fb720001470 con 0x7fb730155860 2023-10-06T01:40:02.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 >> v1:172.21.15.98:6789/0 conn(0x7fb730155860 legacy=0x7fb7300ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fe6fac07700 1 -- 172.21.15.98:0/985167625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f4067840 msgr2=0x7fe6f40c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fe6fac07700 1 --2- 172.21.15.98:0/985167625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f4067840 0x7fe6f40c7a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80098e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb718003ae0 0x7fb718003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7180052e0 0x7fb71800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300acce0 con 0x7fb7180052e0 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c7200 con 0x7fb718003ae0 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71800c280 con 0x7fb718004810 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.689+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72effd700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb718003ae0 0x7fb718003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.697 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72f7fe700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7180052e0 0x7fb71800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 -- 172.21.15.98:0/1689655008 shutdown_connections 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/1689655008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc35806d920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 -- 172.21.15.98:0/1689655008 >> 172.21.15.98:0/1689655008 conn(0x7fc3580680c0 msgr2=0x7fc3580cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 -- 172.21.15.98:0/1689655008 shutdown_connections 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fc35e7cd700 1 -- 172.21.15.98:0/1689655008 wait complete. 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72e7fc700 1 -- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb718003ae0 msgr2=0x7fb718003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb718003ae0 0x7fb718003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72e7fc700 1 -- 172.21.15.98:0/533614625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7180052e0 msgr2=0x7fb71800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7180052e0 0x7fb71800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fc35e7cd700 1 Processor -- start 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72e7fc700 1 -- 172.21.15.98:0/533614625 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300c74e0 con 0x7fb718004810 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.690+0000 7fb72f7fe700 1 -- 172.21.15.98:0/533614625 reap_dead start 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb718004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb72400f340 tx=0x7fb72400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c688afe355875cc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724010040 con 0x7fb718004810 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400fb90 con 0x7fb718004810 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fb72dffb700 1 -- 172.21.15.98:0/533614625 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724017480 con 0x7fb718004810 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fb735810700 1 -- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 msgr2=0x7fb718004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fb735810700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb718004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb72400f340 tx=0x7fb72400d040 comp rx=0 tx=0).stop 2023-10-06T01:40:02.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fb735810700 1 -- 172.21.15.98:0/533614625 shutdown_connections 2023-10-06T01:40:02.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fb735810700 1 --2- 172.21.15.98:0/533614625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb718004c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fb735810700 1 -- 172.21.15.98:0/533614625 >> 172.21.15.98:0/533614625 conn(0x7fb7300680c0 msgr2=0x7fb7300cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.692+0000 7fc35e7cd700 1 -- start start 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc3580c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358168b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 0x7fc358169090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358167bd0 con 0x7fc35806d550 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c2da0 con 0x7fc3580c8320 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35e7cd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc358166fc0 con 0x7fc3580878f0 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35cd6a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 0x7fc358169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35cd6a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 0x7fc358169090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60346/0 (socket says 172.21.15.98:60346) 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc35cd6a700 1 -- 172.21.15.98:0/2471799653 learned_addr learned my addr 172.21.15.98:0/2471799653 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.693+0000 7fc357fff700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc3580c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc3577fe700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 -- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 msgr2=0x7fc358168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 -- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 msgr2=0x7fc358169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 0x7fc358169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 -- 172.21.15.98:0/2471799653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34800d040 con 0x7fc35806d550 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc357fff700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc3580c2860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc348018000 tx=0x7fc34801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348020070 con 0x7fc35806d550 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3480137f0 con 0x7fc35806d550 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480256c0 con 0x7fc35806d550 2023-10-06T01:40:02.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.694+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3581672a0 con 0x7fc35806d550 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.695+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc358167860 con 0x7fc35806d550 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.696+0000 7fe6fac07700 1 -- 172.21.15.98:0/985167625 shutdown_connections 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.696+0000 7fe6fac07700 1 --2- 172.21.15.98:0/985167625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f4067840 0x7fe6f40c7a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.696+0000 7fe6fac07700 1 -- 172.21.15.98:0/985167625 >> 172.21.15.98:0/985167625 conn(0x7fe6f40cdef0 msgr2=0x7fe6f40c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.697+0000 7fe6fac07700 1 -- 172.21.15.98:0/985167625 shutdown_connections 2023-10-06T01:40:02.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.697+0000 7fe6fac07700 1 -- 172.21.15.98:0/985167625 wait complete. 2023-10-06T01:40:02.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc348025af0 con 0x7fc35806d550 2023-10-06T01:40:02.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fb735810700 1 -- 172.21.15.98:0/533614625 shutdown_connections 2023-10-06T01:40:02.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fe6fac07700 1 Processor -- start 2023-10-06T01:40:02.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.699+0000 7fe6fac07700 1 -- start start 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 0x7fe6f4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f40c7fa0 0x7fe6f41610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 0x7fe6f4161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4153fd0 con 0x7fe6f40c7fa0 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4153120 con 0x7fe6f4067840 2023-10-06T01:40:02.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6fac07700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4154460 con 0x7fe6f40caa30 2023-10-06T01:40:02.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fb735810700 1 -- 172.21.15.98:0/533614625 wait complete. 2023-10-06T01:40:02.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fb735810700 1 Processor -- start 2023-10-06T01:40:02.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fc33ffff700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 0x7fc344069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.698+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc34800c050 con 0x7fc35806d550 2023-10-06T01:40:02.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.699+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3380052b0 con 0x7fc35806d550 2023-10-06T01:40:02.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.699+0000 7fc3577fe700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 0x7fc344069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.699+0000 7fc3577fe700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 0x7fc344069460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3580ad990 tx=0x7fc34c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fb735810700 1 -- start start 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb7300b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7300c1b20 0x7fb730157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c13f0 con 0x7fb730168b80 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c0f60 con 0x7fb7300c1b20 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb735810700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301696c0 con 0x7fb718004810 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41802/0 (socket says 172.21.15.98:41802) 2023-10-06T01:40:02.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 -- 172.21.15.98:0/252275438 learned_addr learned my addr 172.21.15.98:0/252275438 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72f7fe700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7300c1b20 0x7fb730157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 -- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 msgr2=0x7fb7300b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb7300b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 -- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7300c1b20 msgr2=0x7fb730157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7300c1b20 0x7fb730157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.702+0000 7fb72e7fc700 1 -- 172.21.15.98:0/252275438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300c0c10 con 0x7fb730168b80 2023-10-06T01:40:02.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.703+0000 7fb72e7fc700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb72400f930 tx=0x7fb72401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6f89a3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 0x7fe6f4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6f89a3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 0x7fe6f4151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60362/0 (socket says 172.21.15.98:60362) 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.700+0000 7fe6f89a3700 1 -- 172.21.15.98:0/4040214507 learned_addr learned my addr 172.21.15.98:0/4040214507 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.703+0000 7fe6f91a4700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 0x7fe6f4161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.703+0000 7fe6f3fff700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f40c7fa0 0x7fe6f41610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 -- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 msgr2=0x7fe6f4151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 0x7fe6f4151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 -- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f40c7fa0 msgr2=0x7fe6f41610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f40c7fa0 0x7fe6f41610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 -- 172.21.15.98:0/4040214507 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009420 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.704+0000 7fe6f91a4700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 0x7fe6f4161620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400cb90 tx=0x7fe6e4001960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.705+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400a7e0 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.705+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400a940 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.705+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ad00 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.707+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f41533c0 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.707+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f4153980 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.708+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f4154140 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.710+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6e4007300 con 0x7fe6f40caa30 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.710+0000 7fe6f0ff9700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 0x7fe6d8069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.710+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe6e40855a0 con 0x7fe6f40caa30 2023-10-06T01:40:02.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.710+0000 7fe6f89a3700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 0x7fe6d8069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.707+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724017480 con 0x7fb730168b80 2023-10-06T01:40:02.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.707+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400b150 con 0x7fb730168b80 2023-10-06T01:40:02.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.707+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724008ea0 con 0x7fb730168b80 2023-10-06T01:40:02.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.711+0000 7fe6f89a3700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 0x7fe6d8069480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80093c0 tx=0x7fe6e8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.711+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc348056730 con 0x7fc35806d550 2023-10-06T01:40:02.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.713+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6e4052520 con 0x7fe6f40caa30 2023-10-06T01:40:02.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.714+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7300c2370 con 0x7fb730168b80 2023-10-06T01:40:02.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.714+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7300c1f90 con 0x7fb730168b80 2023-10-06T01:40:02.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.716+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb72400b760 con 0x7fb730168b80 2023-10-06T01:40:02.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.717+0000 7fb716ffd700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.717+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb724086f20 con 0x7fb730168b80 2023-10-06T01:40:02.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.718+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7100052b0 con 0x7fb730168b80 2023-10-06T01:40:02.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.723+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb724053ca0 con 0x7fb730168b80 2023-10-06T01:40:02.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.727+0000 7fb72effd700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.727+0000 7fb72effd700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb720007f10 tx=0x7fb720007940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.733+0000 7f979addf700 1 Processor -- start 2023-10-06T01:40:02.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.733+0000 7f979addf700 1 -- start start 2023-10-06T01:40:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f979addf700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9794067840 0x7f97940c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f979addf700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979406bfe0 con 0x7f97940caa30 2023-10-06T01:40:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f979addf700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9794162080 con 0x7f97940c7fa0 2023-10-06T01:40:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f979addf700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9794162360 con 0x7f9794067840 2023-10-06T01:40:02.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f9793fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f97940caa30 0x7f979406c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43098/0 (socket says 172.21.15.98:43098) 2023-10-06T01:40:02.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f9793fff700 1 -- 172.21.15.98:0/3753459819 learned_addr learned my addr 172.21.15.98:0/3753459819 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f9798b7b700 1 --2- 172.21.15.98:0/3753459819 >> v2:172.21.15.184:3300/0 conn(0x7f9794067840 0x7f97940c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768197881 0 0) 0x7f9794162080 con 0x7f97940c7fa0 2023-10-06T01:40:02.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9784003570 con 0x7f97940c7fa0 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94190095 0 0) 0x7f979406bfe0 con 0x7f97940caa30 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.734+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9794162080 con 0x7f97940caa30 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f9798b7b700 1 -- 172.21.15.98:0/3753459819 >> v1:172.21.15.184:6789/0 conn(0x7f97940c7fa0 legacy=0x7f9794160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f9798b7b700 1 -- 172.21.15.98:0/3753459819 >> v1:172.21.15.98:6789/0 conn(0x7f97940caa30 legacy=0x7f979406c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f9798b7b700 1 -- 172.21.15.98:0/3753459819 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794162640 con 0x7f9794067840 2023-10-06T01:40:02.737 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f9798b7b700 1 --2- 172.21.15.98:0/3753459819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9794067840 0x7f97940c7a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97880098e0 tx=0x7f97880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3714d50c950bb6e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788011630 con 0x7f9794067840 2023-10-06T01:40:02.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788011790 con 0x7f9794067840 2023-10-06T01:40:02.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.735+0000 7f97937fe700 1 -- 172.21.15.98:0/3753459819 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788011b30 con 0x7f9794067840 2023-10-06T01:40:02.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 -- 172.21.15.98:0/3753459819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9794067840 msgr2=0x7f97940c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 --2- 172.21.15.98:0/3753459819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9794067840 0x7f97940c7a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97880098e0 tx=0x7f97880091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 -- 172.21.15.98:0/3753459819 shutdown_connections 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 --2- 172.21.15.98:0/3753459819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9794067840 0x7f97940c7a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 -- 172.21.15.98:0/3753459819 >> 172.21.15.98:0/3753459819 conn(0x7f97940cdef0 msgr2=0x7f97940c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 -- 172.21.15.98:0/3753459819 shutdown_connections 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.736+0000 7f979addf700 1 -- 172.21.15.98:0/3753459819 wait complete. 2023-10-06T01:40:02.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.738+0000 7f979addf700 1 Processor -- start 2023-10-06T01:40:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.738+0000 7f979addf700 1 -- start start 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9794067840 0x7f9794147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97940caa30 0x7f979406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979416eae0 con 0x7f9794067840 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979415e900 con 0x7f97940caa30 2023-10-06T01:40:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.739+0000 7f979addf700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979415ebe0 con 0x7f97940c7fa0 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:43908/0 (socket says 172.21.15.98:43908) 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 -- 172.21.15.98:0/3371436712 learned_addr learned my addr 172.21.15.98:0/3371436712 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 -- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97940caa30 msgr2=0x7f979406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97940caa30 0x7f979406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 -- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9794067840 msgr2=0x7f9794147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9794067840 0x7f9794147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 -- 172.21.15.98:0/3371436712 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9788009420 con 0x7f97940c7fa0 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.740+0000 7f9793fff700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f978000c560 tx=0x7f9780000fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.741+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780005b30 con 0x7f97940c7fa0 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.741+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9780005cb0 con 0x7f97940c7fa0 2023-10-06T01:40:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.741+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780007320 con 0x7f97940c7fa0 2023-10-06T01:40:02.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.741+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979415eec0 con 0x7f97940c7fa0 2023-10-06T01:40:02.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.742+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979415e520 con 0x7f97940c7fa0 2023-10-06T01:40:02.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.742+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979415f0e0 con 0x7f97940c7fa0 2023-10-06T01:40:02.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.743+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f978001c440 con 0x7f97940c7fa0 2023-10-06T01:40:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.744+0000 7f9790ff9700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 0x7f97780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.744+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9780084280 con 0x7f97940c7fa0 2023-10-06T01:40:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.744+0000 7f9798b7b700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 0x7f97780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.744+0000 7f9798b7b700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 0x7f97780693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9788003080 tx=0x7f9788016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.747+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9780051080 con 0x7f97940c7fa0 2023-10-06T01:40:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.748+0000 7fda2a1c5700 1 Processor -- start 2023-10-06T01:40:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.748+0000 7fda2a1c5700 1 -- start start 2023-10-06T01:40:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.750+0000 7fda2a1c5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda24159a60 0x7fda24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.750+0000 7fda2a1c5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.750+0000 7fda2a1c5700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda24069f00 con 0x7fda24069aa0 2023-10-06T01:40:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.750+0000 7fda2a1c5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2415a220 con 0x7fda24159a60 2023-10-06T01:40:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.750+0000 7fda2a1c5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2415b960 con 0x7fda2415daf0 2023-10-06T01:40:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.752+0000 7fda22ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fda24069aa0 0x7fda2415d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43100/0 (socket says 172.21.15.98:43100) 2023-10-06T01:40:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.752+0000 7fda22ffd700 1 -- 172.21.15.98:0/175770051 learned_addr learned my addr 172.21.15.98:0/175770051 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda23fff700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda237fe700 1 --2- 172.21.15.98:0/175770051 >> v2:172.21.15.98:3300/0 conn(0x7fda24159a60 0x7fda24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fba55319700 1 Processor -- start 2023-10-06T01:40:02.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda237fe700 1 -- 172.21.15.98:0/175770051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda2415daf0 msgr2=0x7fda2415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda237fe700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda237fe700 1 -- 172.21.15.98:0/175770051 >> v1:172.21.15.98:6789/0 conn(0x7fda24069aa0 legacy=0x7fda2415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fda237fe700 1 -- 172.21.15.98:0/175770051 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2415e2f0 con 0x7fda24159a60 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.754+0000 7fda237fe700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda24159a60 0x7fda24069560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda180098e0 tx=0x7fda180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186357f562ba6822 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.754+0000 7fda227fc700 1 -- 172.21.15.98:0/175770051 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18009e80 con 0x7fda24159a60 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.753+0000 7fba55319700 1 -- start start 2023-10-06T01:40:02.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.754+0000 7fda227fc700 1 -- 172.21.15.98:0/175770051 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1800b860 con 0x7fda24159a60 2023-10-06T01:40:02.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.754+0000 7fda227fc700 1 -- 172.21.15.98:0/175770051 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800bc20 con 0x7fda24159a60 2023-10-06T01:40:02.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba5015d8f0 0x7fba5015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 -- 172.21.15.98:0/175770051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda24159a60 msgr2=0x7fda24069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda24159a60 0x7fda24069560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda180098e0 tx=0x7fda180091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba500b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba50068600 0x7fba50068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500c8ef0 con 0x7fba5015d8f0 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500b2700 con 0x7fba50068600 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba55319700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba50068fb0 con 0x7fba500b1d50 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 -- 172.21.15.98:0/175770051 shutdown_connections 2023-10-06T01:40:02.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/175770051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda24159a60 0x7fda24069560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 -- 172.21.15.98:0/175770051 >> 172.21.15.98:0/175770051 conn(0x7fda240680c0 msgr2=0x7fda240cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba4effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba5015d8f0 0x7fba5015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba4effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba5015d8f0 0x7fba5015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41836/0 (socket says 172.21.15.98:41836) 2023-10-06T01:40:02.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fba4effd700 1 -- 172.21.15.98:0/2251770992 learned_addr learned my addr 172.21.15.98:0/2251770992 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 -- 172.21.15.98:0/175770051 shutdown_connections 2023-10-06T01:40:02.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.755+0000 7fda2a1c5700 1 -- 172.21.15.98:0/175770051 wait complete. 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 -- 172.21.15.98:0/2251770992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 msgr2=0x7fba500b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 --2- 172.21.15.98:0/2251770992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba500b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 -- 172.21.15.98:0/2251770992 >> v2:172.21.15.184:3300/0 conn(0x7fba50068600 msgr2=0x7fba50068a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 --2- 172.21.15.98:0/2251770992 >> v2:172.21.15.184:3300/0 conn(0x7fba50068600 0x7fba50068a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 -- 172.21.15.98:0/2251770992 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500c9750 con 0x7fba5015d8f0 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.756+0000 7fba4effd700 1 --2- 172.21.15.98:0/2251770992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015dcc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba500c8ea0 tx=0x7fba44009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b190044205e8558 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.757+0000 7fda2a1c5700 1 Processor -- start 2023-10-06T01:40:02.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.757+0000 7fda2a1c5700 1 -- start start 2023-10-06T01:40:02.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fba4dffb700 1 -- 172.21.15.98:0/2251770992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44005c10 con 0x7fba5015d8f0 2023-10-06T01:40:02.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fba4dffb700 1 -- 172.21.15.98:0/2251770992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba44005d70 con 0x7fba5015d8f0 2023-10-06T01:40:02.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fba4dffb700 1 -- 172.21.15.98:0/2251770992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400b880 con 0x7fba5015d8f0 2023-10-06T01:40:02.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 0x7fda2416cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24159a60 0x7fda240b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240bfef0 con 0x7fda2415daf0 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240b0910 con 0x7fda24069aa0 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda2a1c5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda240b0bf0 con 0x7fda24159a60 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41846/0 (socket says 172.21.15.98:41846) 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda23fff700 1 -- 172.21.15.98:0/419238708 learned_addr learned my addr 172.21.15.98:0/419238708 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.758+0000 7fda237fe700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 0x7fda2416cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda22ffd700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24159a60 0x7fda240b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fba55319700 1 -- 172.21.15.98:0/2251770992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 msgr2=0x7fba5015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fba55319700 1 --2- 172.21.15.98:0/2251770992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015dcc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba500c8ea0 tx=0x7fba44009220 comp rx=0 tx=0).stop 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 -- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24159a60 msgr2=0x7fda240b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24159a60 0x7fda240b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 -- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 msgr2=0x7fda2415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 -- 172.21.15.98:0/419238708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda18009420 con 0x7fda24069aa0 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda23fff700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.759+0000 7fda237fe700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 0x7fda2416cb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda18016000 tx=0x7fda18017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.760+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1801f070 con 0x7fda24069aa0 2023-10-06T01:40:02.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.760+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1800b9d0 con 0x7fda24069aa0 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.760+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda240b0ed0 con 0x7fda24069aa0 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.760+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18011420 con 0x7fda24069aa0 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.760+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2416d090 con 0x7fda24069aa0 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.761+0000 7fba55319700 1 -- 172.21.15.98:0/2251770992 shutdown_connections 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.761+0000 7fba55319700 1 --2- 172.21.15.98:0/2251770992 >> v2:172.21.15.184:3300/0 conn(0x7fba50068600 0x7fba50068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.761+0000 7fba55319700 1 --2- 172.21.15.98:0/2251770992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015dcc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.761+0000 7fba55319700 1 --2- 172.21.15.98:0/2251770992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba500b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.761+0000 7fba55319700 1 -- 172.21.15.98:0/2251770992 >> 172.21.15.98:0/2251770992 conn(0x7fba5006de80 msgr2=0x7fba5006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.762+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda240b0d30 con 0x7fda24069aa0 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.762+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda180117f0 con 0x7fda24069aa0 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.762+0000 7fda0b7fe700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 0x7fda10069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.762+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fda18085bd0 con 0x7fda24069aa0 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.762+0000 7fda22ffd700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 0x7fda10069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.763+0000 7fda22ffd700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 0x7fda10069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda2415d4b0 tx=0x7fda14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.767+0000 7fa42d719700 1 Processor -- start 2023-10-06T01:40:02.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.767+0000 7fba55319700 1 -- 172.21.15.98:0/2251770992 shutdown_connections 2023-10-06T01:40:02.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.767+0000 7fa42d719700 1 -- start start 2023-10-06T01:40:02.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.767+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda240b0d30 con 0x7fda24069aa0 2023-10-06T01:40:02.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.768+0000 7fa42d719700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 0x7fa4280ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.768+0000 7fa42d719700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa428069560 0x7fa4280699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.768+0000 7fa42d719700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42806d5c0 con 0x7fa42815a940 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.768+0000 7fa42d719700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280aeb10 con 0x7fa428069560 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.768+0000 7fa42d719700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa428069f20 con 0x7fa42815ad10 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa426ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa42815a940 0x7fa4280ad9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43116/0 (socket says 172.21.15.98:43116) 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa426ffd700 1 -- 172.21.15.98:0/3685725983 learned_addr learned my addr 172.21.15.98:0/3685725983 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4267fc700 1 --2- 172.21.15.98:0/3685725983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 0x7fa4280ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.767+0000 7fba55319700 1 -- 172.21.15.98:0/2251770992 wait complete. 2023-10-06T01:40:02.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fba55319700 1 Processor -- start 2023-10-06T01:40:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4277fe700 1 --2- 172.21.15.98:0/3685725983 >> v2:172.21.15.184:3300/0 conn(0x7fa428069560 0x7fa4280699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4267fc700 1 -- 172.21.15.98:0/3685725983 >> v2:172.21.15.184:3300/0 conn(0x7fa428069560 msgr2=0x7fa4280699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4267fc700 1 --2- 172.21.15.98:0/3685725983 >> v2:172.21.15.184:3300/0 conn(0x7fa428069560 0x7fa4280699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4267fc700 1 -- 172.21.15.98:0/3685725983 >> v1:172.21.15.98:6789/0 conn(0x7fa42815a940 legacy=0x7fa4280ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fa4267fc700 1 -- 172.21.15.98:0/3685725983 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280ae290 con 0x7fa42815ad10 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.770+0000 7fa4267fc700 1 --2- 172.21.15.98:0/3685725983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 0x7fa4280ac110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098e0 tx=0x7fa4180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec286bf672f21d21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.770+0000 7fa425ffb700 1 -- 172.21.15.98:0/3685725983 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418009e80 con 0x7fa42815ad10 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.770+0000 7fa425ffb700 1 -- 172.21.15.98:0/3685725983 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800b760 con 0x7fa42815ad10 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.770+0000 7fa425ffb700 1 -- 172.21.15.98:0/3685725983 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41800bb00 con 0x7fa42815ad10 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 -- 172.21.15.98:0/3685725983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 msgr2=0x7fa4280ac110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 --2- 172.21.15.98:0/3685725983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 0x7fa4280ac110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098e0 tx=0x7fa4180091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.769+0000 7fba55319700 1 -- start start 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.770+0000 7fba55319700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba55319700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba5015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba55319700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba55319700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba500a9250 con 0x7fba5015d8f0 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba55319700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015f180 con 0x7fba50068600 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba55319700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015f3e0 con 0x7fba500b1d50 2023-10-06T01:40:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba4effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba4effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60418/0 (socket says 172.21.15.98:60418) 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba4effd700 1 -- 172.21.15.98:0/3653280378 learned_addr learned my addr 172.21.15.98:0/3653280378 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fba4f7fe700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 -- 172.21.15.98:0/3685725983 shutdown_connections 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 --2- 172.21.15.98:0/3685725983 >> v2:172.21.15.184:3300/0 conn(0x7fa428069560 0x7fa4280699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 --2- 172.21.15.98:0/3685725983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa42815ad10 0x7fa4280ac110 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098e0 tx=0x7fa4180091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.771+0000 7fa42d719700 1 -- 172.21.15.98:0/3685725983 >> 172.21.15.98:0/3685725983 conn(0x7fa4280680c0 msgr2=0x7fa4280cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4e7fc700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba5015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 -- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 msgr2=0x7fba5015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba5015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 -- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 msgr2=0x7fba5015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 -- 172.21.15.98:0/3653280378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009480 con 0x7fba50068600 2023-10-06T01:40:02.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4e7fc700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba5015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:02.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba4effd700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba5006d2a0 tx=0x7fba44016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fa42d719700 1 -- 172.21.15.98:0/3685725983 shutdown_connections 2023-10-06T01:40:02.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.774+0000 7ff5779ec700 1 Processor -- start 2023-10-06T01:40:02.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44006130 con 0x7fba50068600 2023-10-06T01:40:02.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba44020070 con 0x7fba50068600 2023-10-06T01:40:02.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44006500 con 0x7fba50068600 2023-10-06T01:40:02.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.774+0000 7ff5779ec700 1 -- start start 2023-10-06T01:40:02.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff5779ec700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff5700cc920 0x7ff57006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff5779ec700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 0x7ff5700c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff5779ec700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57006c9e0 con 0x7ff570156390 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff5779ec700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff570156b60 con 0x7ff5700cc920 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff5779ec700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff570156e40 con 0x7ff57006c1a0 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff575788700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff570156390 0x7ff5700cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:39250/0 (socket says 172.21.15.98:39250) 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7ff575788700 1 -- 172.21.15.98:0/335063068 learned_addr learned my addr 172.21.15.98:0/335063068 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:02.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 --2- 172.21.15.98:0/335063068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 0x7ff5700c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff574f87700 1 --2- 172.21.15.98:0/335063068 >> v2:172.21.15.98:3300/0 conn(0x7ff5700cc920 0x7ff57006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff563fff700 1 -- 172.21.15.98:0/335063068 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110070751 0 0) 0x7ff57006c9e0 con 0x7ff570156390 2023-10-06T01:40:02.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff563fff700 1 -- 172.21.15.98:0/335063068 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff55c003570 con 0x7ff570156390 2023-10-06T01:40:02.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.774+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5015f6c0 con 0x7fba50068600 2023-10-06T01:40:02.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.775+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5015fc80 con 0x7fba50068600 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 -- 172.21.15.98:0/335063068 >> v2:172.21.15.98:3300/0 conn(0x7ff5700cc920 msgr2=0x7ff57006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 --2- 172.21.15.98:0/335063068 >> v2:172.21.15.98:3300/0 conn(0x7ff5700cc920 0x7ff57006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 -- 172.21.15.98:0/335063068 >> v1:172.21.15.184:6789/0 conn(0x7ff570156390 legacy=0x7ff5700cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 -- 172.21.15.98:0/335063068 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff570157120 con 0x7ff57006c1a0 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff575f89700 1 --2- 172.21.15.98:0/335063068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 0x7ff5700c88c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff568009e60 tx=0x7ff568009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc043008fe0990 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff563fff700 1 -- 172.21.15.98:0/335063068 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56800e070 con 0x7ff57006c1a0 2023-10-06T01:40:02.785 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.772+0000 7fa42d719700 1 -- 172.21.15.98:0/3685725983 wait complete. 2023-10-06T01:40:02.786 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.774+0000 7fa42d719700 1 Processor -- start 2023-10-06T01:40:02.786 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7fa42d719700 1 -- start start 2023-10-06T01:40:02.786 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.776+0000 7ff563fff700 1 -- 172.21.15.98:0/335063068 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56800bab0 con 0x7ff57006c1a0 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff563fff700 1 -- 172.21.15.98:0/335063068 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56800be30 con 0x7ff57006c1a0 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 -- 172.21.15.98:0/335063068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 msgr2=0x7ff5700c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 --2- 172.21.15.98:0/335063068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 0x7ff5700c88c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff568009e60 tx=0x7ff568009780 comp rx=0 tx=0).stop 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 -- 172.21.15.98:0/335063068 shutdown_connections 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 --2- 172.21.15.98:0/335063068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57006c1a0 0x7ff5700c88c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 --2- 172.21.15.98:0/335063068 >> v2:172.21.15.98:3300/0 conn(0x7ff5700cc920 0x7ff57006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 -- 172.21.15.98:0/335063068 >> 172.21.15.98:0/335063068 conn(0x7ff5700cdef0 msgr2=0x7ff5700c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7ff5779ec700 1 -- 172.21.15.98:0/335063068 shutdown_connections 2023-10-06T01:40:02.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa428069560 0x7fa4280caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa42818aae0 0x7fa42814bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281522c0 con 0x7fa42815a940 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281516b0 con 0x7fa42818aae0 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.777+0000 7fa42d719700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa428151990 con 0x7fa428069560 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41882/0 (socket says 172.21.15.98:41882) 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 -- 172.21.15.98:0/706191526 learned_addr learned my addr 172.21.15.98:0/706191526 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4277fe700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa42818aae0 0x7fa42814bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 -- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa428069560 msgr2=0x7fa4280caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa428069560 0x7fa4280caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 -- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa42818aae0 msgr2=0x7fa42814bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa42818aae0 0x7fa42814bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7fa4267fc700 1 -- 172.21.15.98:0/706191526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009420 con 0x7fa42815a940 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba44022030 con 0x7fba50068600 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.778+0000 7ff5779ec700 1 -- 172.21.15.98:0/335063068 wait complete. 2023-10-06T01:40:02.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7ff5779ec700 1 Processor -- start 2023-10-06T01:40:02.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7fba36ffd700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fba440864a0 con 0x7fba50068600 2023-10-06T01:40:02.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7ff5779ec700 1 -- start start 2023-10-06T01:40:02.790 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.779+0000 7fa4267fc700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa418002000 tx=0x7fa418016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 0x7ff5700c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 0x7ff5700c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700c5420 con 0x7ff5700cc920 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700c56c0 con 0x7ff57006c1a0 2023-10-06T01:40:02.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7ff5779ec700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700c59a0 con 0x7ff570156390 2023-10-06T01:40:02.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.780+0000 7fba4e7fc700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff575f89700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff575f89700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:43988/0 (socket says 172.21.15.98:43988) 2023-10-06T01:40:02.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff575f89700 1 -- 172.21.15.98:0/4022927970 learned_addr learned my addr 172.21.15.98:0/4022927970 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:02.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 0x7ff5700c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7fba4e7fc700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 0x7fba3c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba500ca0e0 tx=0x7fba400091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff575788700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 0x7ff5700c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 -- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 msgr2=0x7ff5700c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 -- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 msgr2=0x7ff5700c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 0x7ff5700c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.781+0000 7ff574f87700 1 -- 172.21.15.98:0/4022927970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568009a20 con 0x7ff5700cc920 2023-10-06T01:40:02.794 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff575f89700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff575788700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 0x7ff5700c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff574f87700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 0x7ff5700c4ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff56400ea20 tx=0x7ff5640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400c7f0 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56400c950 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5700c5c80 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400ccf0 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.782+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5700c6240 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.784+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff564013680 con 0x7ff5700cc920 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.785+0000 7ff5617fa700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 0x7ff55c069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.785+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff564084b90 con 0x7ff5700cc920 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.787+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300052b0 con 0x7fba50068600 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41800bae0 con 0x7fa42815a940 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa428151c70 con 0x7fa42815a940 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41801a460 con 0x7fa42815a940 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa428188750 con 0x7fa42815a940 2023-10-06T01:40:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41801a800 con 0x7fa42815a940 2023-10-06T01:40:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.792+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba44053220 con 0x7fba50068600 2023-10-06T01:40:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.791+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff570064ad0 con 0x7ff5700cc920 2023-10-06T01:40:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.792+0000 7ff575788700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 0x7ff55c069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.793+0000 7ff575788700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 0x7ff55c069830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0058d0 tx=0x7ff56c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.794+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa418011420 con 0x7fa42815a940 2023-10-06T01:40:02.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.796+0000 7fa40effd700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 0x7fa4140694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:02.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.796+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa418085fe0 con 0x7fa42815a940 2023-10-06T01:40:02.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.796+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4080052b0 con 0x7fa42815a940 2023-10-06T01:40:02.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.797+0000 7fa426ffd700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 0x7fa4140694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:02.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.797+0000 7fa426ffd700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 0x7fa4140694b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0058d0 tx=0x7fa41c007ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:02.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.803+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5640519b0 con 0x7ff5700cc920 2023-10-06T01:40:02.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:02.803+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa418052ee0 con 0x7fa42815a940 2023-10-06T01:40:03.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.091+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe6f4064ad0 con 0x7fe6f40caa30 2023-10-06T01:40:03.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.094+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc338005560 con 0x7fc35806d550 2023-10-06T01:40:03.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.095+0000 7fe6f0ff9700 1 -- 172.21.15.98:0/4040214507 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe6e405b020 con 0x7fe6f40caa30 2023-10-06T01:40:03.097 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:03.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.095+0000 7fc33ffff700 1 -- 172.21.15.98:0/2471799653 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc34802a030 con 0x7fc35806d550 2023-10-06T01:40:03.099 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:03.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.098+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 msgr2=0x7fe6d8069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.098+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 0x7fe6d8069480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80093c0 tx=0x7fe6e8016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.098+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 msgr2=0x7fe6f4161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.098+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 0x7fe6f4161620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400cb90 tx=0x7fe6e4001960 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 shutdown_connections 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe6d8066fd0 0x7fe6d8069480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6f40caa30 0x7fe6f4161620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f4067840 0x7fe6f4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 --2- 172.21.15.98:0/4040214507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f40c7fa0 0x7fe6f41610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 >> 172.21.15.98:0/4040214507 conn(0x7fe6f40cdef0 msgr2=0x7fe6f40c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 msgr2=0x7fc344069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 0x7fc344069460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3580ad990 tx=0x7fc34c009040 comp rx=0 tx=0).stop 2023-10-06T01:40:03.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 msgr2=0x7fc3580c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.099+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc3580c2860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc348018000 tx=0x7fc34801b000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.107+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 shutdown_connections 2023-10-06T01:40:03.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.108+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb710005560 con 0x7fb730168b80 2023-10-06T01:40:03.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.109+0000 7fb716ffd700 1 -- 172.21.15.98:0/252275438 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7240572c0 con 0x7fb730168b80 2023-10-06T01:40:03.110 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 shutdown_connections 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc344066fb0 0x7fc344069460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3580c8320 0x7fc358169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35806d550 0x7fc3580c2860 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 --2- 172.21.15.98:0/2471799653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3580878f0 0x7fc358168b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 >> 172.21.15.98:0/2471799653 conn(0x7fc3580680c0 msgr2=0x7fc3580b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.112 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.110+0000 7fe6fac07700 1 -- 172.21.15.98:0/4040214507 wait complete. 2023-10-06T01:40:03.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 msgr2=0x7fb71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb720007f10 tx=0x7fb720007940 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 msgr2=0x7fb7300c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb72400f930 tx=0x7fb72401e000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 shutdown_connections 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7300c1b20 0x7fb730157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb718004810 0x7fb7300b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 --2- 172.21.15.98:0/252275438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb730168b80 0x7fb7300c15e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 >> 172.21.15.98:0/252275438 conn(0x7fb7300680c0 msgr2=0x7fb7300ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 shutdown_connections 2023-10-06T01:40:03.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.116+0000 7fb735810700 1 -- 172.21.15.98:0/252275438 wait complete. 2023-10-06T01:40:03.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.126+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 shutdown_connections 2023-10-06T01:40:03.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.126+0000 7fc35e7cd700 1 -- 172.21.15.98:0/2471799653 wait complete. 2023-10-06T01:40:03.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-06T01:40:03.130 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.130+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9794064ad0 con 0x7f97940c7fa0 2023-10-06T01:40:03.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-06T01:40:03.136 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.135+0000 7f9790ff9700 1 -- 172.21.15.98:0/3371436712 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9780017090 con 0x7f97940c7fa0 2023-10-06T01:40:03.136 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:03.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 msgr2=0x7f97780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 0x7f97780693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9788003080 tx=0x7f9788016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 msgr2=0x7f979406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f978000c560 tx=0x7f9780000fe0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 shutdown_connections 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9778066f30 0x7f97780693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97940caa30 0x7f979406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9794067840 0x7f9794147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 --2- 172.21.15.98:0/3371436712 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97940c7fa0 0x7f979406e5c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 >> 172.21.15.98:0/3371436712 conn(0x7f97940cdef0 msgr2=0x7f979415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 shutdown_connections 2023-10-06T01:40:03.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.138+0000 7f979addf700 1 -- 172.21.15.98:0/3371436712 wait complete. 2023-10-06T01:40:03.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-06T01:40:03.147 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.150+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fda240b0d30 con 0x7fda24069aa0 2023-10-06T01:40:03.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-10-06T01:40:03.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.151+0000 7fda0b7fe700 1 -- 172.21.15.98:0/419238708 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda18015020 con 0x7fda24069aa0 2023-10-06T01:40:03.153 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:03.153 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 msgr2=0x7fda10069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 0x7fda10069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda2415d4b0 tx=0x7fda14009040 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 msgr2=0x7fda2416cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 0x7fda2416cb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda18016000 tx=0x7fda18017000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 shutdown_connections 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fda10066f60 0x7fda10069410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2415daf0 0x7fda2415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda24069aa0 0x7fda2416cb50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 --2- 172.21.15.98:0/419238708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24159a60 0x7fda240b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 >> 172.21.15.98:0/419238708 conn(0x7fda240680c0 msgr2=0x7fda2406d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 shutdown_connections 2023-10-06T01:40:03.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.153+0000 7fda2a1c5700 1 -- 172.21.15.98:0/419238708 wait complete. 2023-10-06T01:40:03.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.158+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fba30005560 con 0x7fba50068600 2023-10-06T01:40:03.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.159+0000 7fba36ffd700 1 -- 172.21.15.98:0/3653280378 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba44056840 con 0x7fba50068600 2023-10-06T01:40:03.160 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 msgr2=0x7fba3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 0x7fba3c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba500ca0e0 tx=0x7fba400091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 msgr2=0x7fba50161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba5006d2a0 tx=0x7fba44016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 shutdown_connections 2023-10-06T01:40:03.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fba3c066f30 0x7fba3c0693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5015d8f0 0x7fba5015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba50068600 0x7fba50161190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 --2- 172.21.15.98:0/3653280378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba500b1d50 0x7fba5015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 >> 172.21.15.98:0/3653280378 conn(0x7fba5006de80 msgr2=0x7fba500cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 shutdown_connections 2023-10-06T01:40:03.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.162+0000 7fba55319700 1 -- 172.21.15.98:0/3653280378 wait complete. 2023-10-06T01:40:03.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-06T01:40:03.168 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T01:40:03.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.179+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff570157400 con 0x7ff5700cc920 2023-10-06T01:40:03.181 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.179+0000 7ff5617fa700 1 -- 172.21.15.98:0/4022927970 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff564017090 con 0x7ff5700cc920 2023-10-06T01:40:03.182 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 msgr2=0x7ff55c069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 0x7ff55c069830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0058d0 tx=0x7ff56c009040 comp rx=0 tx=0).stop 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 msgr2=0x7ff5700c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 0x7ff5700c4ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff56400ea20 tx=0x7ff5640091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 shutdown_connections 2023-10-06T01:40:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff55c067380 0x7ff55c069830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff570156390 0x7ff5700c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57006c1a0 0x7ff5700c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 --2- 172.21.15.98:0/4022927970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5700cc920 0x7ff5700c4ee0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 >> 172.21.15.98:0/4022927970 conn(0x7ff5700cdef0 msgr2=0x7ff5701537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 shutdown_connections 2023-10-06T01:40:03.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.182+0000 7ff5779ec700 1 -- 172.21.15.98:0/4022927970 wait complete. 2023-10-06T01:40:03.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.184+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa408005830 con 0x7fa42815a940 2023-10-06T01:40:03.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.184+0000 7fa40effd700 1 -- 172.21.15.98:0/706191526 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa418056500 con 0x7fa42815a940 2023-10-06T01:40:03.186 INFO:teuthology.orchestra.run.smithi098.stdout:55834574857 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 msgr2=0x7fa4140694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 0x7fa4140694b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0058d0 tx=0x7fa41c007ac0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 msgr2=0x7fa428187ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa418002000 tx=0x7fa418016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 shutdown_connections 2023-10-06T01:40:03.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa414067000 0x7fa4140694b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa42818aae0 0x7fa42814bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa428069560 0x7fa4280caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 --2- 172.21.15.98:0/706191526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa42815a940 0x7fa428187ca0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 >> 172.21.15.98:0/706191526 conn(0x7fa4280680c0 msgr2=0x7fa4280cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 shutdown_connections 2023-10-06T01:40:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.186+0000 7fa42d719700 1 -- 172.21.15.98:0/706191526 wait complete. 2023-10-06T01:40:03.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-10-06T01:40:03.197 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-10-06T01:40:03.200 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:03.200 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T01:40:03.201 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:03.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.355+0000 7f92a1c3b700 1 Processor -- start 2023-10-06T01:40:03.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.355+0000 7f92a1c3b700 1 -- start start 2023-10-06T01:40:03.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f929c0b2620 0x7f929c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c0cbfe0 con 0x7f929c0b29f0 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c0cb3d0 con 0x7f929c0b2620 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0cb6b0 con 0x7f929c159870 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f929c0b2620 0x7f929c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f929c0b2620 0x7f929c0b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41900/0 (socket says 172.21.15.98:41900) 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 -- 172.21.15.98:0/1613869699 learned_addr learned my addr 172.21.15.98:0/1613869699 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929bfff700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 -- 172.21.15.98:0/1613869699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 msgr2=0x7f929c0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 -- 172.21.15.98:0/1613869699 >> v1:172.21.15.184:6789/0 conn(0x7f929c0b29f0 legacy=0x7f929c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 -- 172.21.15.98:0/1613869699 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0cb990 con 0x7f929c0b2620 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929bfff700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929b7fe700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b2620 0x7f929c0b06d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f928c009990 tx=0x7f928c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d56fefe5fa44d97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929a7fc700 1 -- 172.21.15.98:0/1613869699 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c005c10 con 0x7f929c0b2620 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929a7fc700 1 -- 172.21.15.98:0/1613869699 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c005d70 con 0x7f929c0b2620 2023-10-06T01:40:03.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f929a7fc700 1 -- 172.21.15.98:0/1613869699 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00b880 con 0x7f929c0b2620 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- 172.21.15.98:0/1613869699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b2620 msgr2=0x7f929c0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b2620 0x7f929c0b06d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f928c009990 tx=0x7f928c009220 comp rx=0 tx=0).stop 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- 172.21.15.98:0/1613869699 shutdown_connections 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/1613869699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b2620 0x7f929c0b06d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- 172.21.15.98:0/1613869699 >> 172.21.15.98:0/1613869699 conn(0x7f929c06ed20 msgr2=0x7f929c06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- 172.21.15.98:0/1613869699 shutdown_connections 2023-10-06T01:40:03.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.356+0000 7f92a1c3b700 1 -- 172.21.15.98:0/1613869699 wait complete. 2023-10-06T01:40:03.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.357+0000 7f92a1c3b700 1 Processor -- start 2023-10-06T01:40:03.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.357+0000 7f92a1c3b700 1 -- start start 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c0b2620 0x7f929c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1584a0 con 0x7f929c0b29f0 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c157820 con 0x7f929c0b2620 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f92a1c3b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c157b00 con 0x7f929c159870 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f929affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f929affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41908/0 (socket says 172.21.15.98:41908) 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f929affd700 1 -- 172.21.15.98:0/725082984 learned_addr learned my addr 172.21.15.98:0/725082984 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f929b7fe700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c0b2620 0x7f929c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.358+0000 7f929bfff700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 -- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 msgr2=0x7f929c169ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c169ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 -- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c0b2620 msgr2=0x7f929c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c0b2620 0x7f929c0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 -- 172.21.15.98:0/725082984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c009480 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f929affd700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f929000c590 tx=0x7f929000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290011590 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290011710 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c157de0 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290011ad0 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.359+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c157480 con 0x7f929c0b29f0 2023-10-06T01:40:03.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.360+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c064ad0 con 0x7f929c0b29f0 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.360+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f929001b3b0 con 0x7f929c0b29f0 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.361+0000 7f92837fe700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.361+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9290083df0 con 0x7f929c0b29f0 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.361+0000 7f929b7fe700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.361+0000 7f929b7fe700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f929c0cfb40 tx=0x7f928c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.365+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9290050c10 con 0x7f929c0b29f0 2023-10-06T01:40:03.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.543+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f929c0cbc70 con 0x7f9288066f30 2023-10-06T01:40:03.554 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.553+0000 7f92837fe700 1 -- 172.21.15.98:0/725082984 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367616 (secure 0 0 0) 0x7f929c0cbc70 con 0x7f9288066f30 2023-10-06T01:40:03.558 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 msgr2=0x7f92880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f929c0cfb40 tx=0x7f928c016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 msgr2=0x7f929c156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f929000c590 tx=0x7f929000c8a0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 shutdown_connections 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159870 0x7f929c169ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c0b2620 0x7f929c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 --2- 172.21.15.98:0/725082984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0b29f0 0x7f929c156f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 >> 172.21.15.98:0/725082984 conn(0x7f929c06ed20 msgr2=0x7f929c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 shutdown_connections 2023-10-06T01:40:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.556+0000 7f92a1c3b700 1 -- 172.21.15.98:0/725082984 wait complete. 2023-10-06T01:40:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:03.570 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-06T01:40:01.788217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.421732"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346761+0000","last_change":"2023-10-06T01:39:45.335803+0000","last_active":"2023-10-06T01:39:49.346761+0000","last_peered":"2023-10-06T01:39:49.346761+0000","last_clean":"2023-10-06T01:39:49.346761+0000","last_became_active":"2023-10-06T01:39:45.335420+0000","last_became_peered":"2023-10-06T01:39:45.335420+0000","last_unstale":"2023-10-06T01:39:49.346761+0000","last_undegraded":"2023-10-06T01:39:49.346761+0000","last_fullsized":"2023-10-06T01:39:49.346761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:58.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353326+0000","last_change":"2023-10-06T01:39:45.336392+0000","last_active":"2023-10-06T01:39:49.353326+0000","last_peered":"2023-10-06T01:39:49.353326+0000","last_clean":"2023-10-06T01:39:49.353326+0000","last_became_active":"2023-10-06T01:39:45.336114+0000","last_became_peered":"2023-10-06T01:39:45.336114+0000","last_unstale":"2023-10-06T01:39:49.353326+0000","last_undegraded":"2023-10-06T01:39:49.353326+0000","last_fullsized":"2023-10-06T01:39:49.353326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340003+0000","last_change":"2023-10-06T01:39:45.330889+0000","last_active":"2023-10-06T01:39:49.340003+0000","last_peered":"2023-10-06T01:39:49.340003+0000","last_clean":"2023-10-06T01:39:49.340003+0000","last_became_active":"2023-10-06T01:39:45.330584+0000","last_became_peered":"2023-10-06T01:39:45.330584+0000","last_unstale":"2023-10-06T01:39:49.340003+0000","last_undegraded":"2023-10-06T01:39:49.340003+0000","last_fullsized":"2023-10-06T01:39:49.340003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:55.095137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340381+0000","last_change":"2023-10-06T01:39:45.327678+0000","last_active":"2023-10-06T01:39:49.340381+0000","last_peered":"2023-10-06T01:39:49.340381+0000","last_clean":"2023-10-06T01:39:49.340381+0000","last_became_active":"2023-10-06T01:39:45.327329+0000","last_became_peered":"2023-10-06T01:39:45.327329+0000","last_unstale":"2023-10-06T01:39:49.340381+0000","last_undegraded":"2023-10-06T01:39:49.340381+0000","last_fullsized":"2023-10-06T01:39:49.340381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:13.570152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347538+0000","last_change":"2023-10-06T01:39:45.342805+0000","last_active":"2023-10-06T01:39:49.347538+0000","last_peered":"2023-10-06T01:39:49.347538+0000","last_clean":"2023-10-06T01:39:49.347538+0000","last_became_active":"2023-10-06T01:39:45.342510+0000","last_became_peered":"2023-10-06T01:39:45.342510+0000","last_unstale":"2023-10-06T01:39:49.347538+0000","last_undegraded":"2023-10-06T01:39:49.347538+0000","last_fullsized":"2023-10-06T01:39:49.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340791+0000","last_change":"2023-10-06T01:39:45.328521+0000","last_active":"2023-10-06T01:39:49.340791+0000","last_peered":"2023-10-06T01:39:49.340791+0000","last_clean":"2023-10-06T01:39:49.340791+0000","last_became_active":"2023-10-06T01:39:45.327249+0000","last_became_peered":"2023-10-06T01:39:45.327249+0000","last_unstale":"2023-10-06T01:39:49.340791+0000","last_undegraded":"2023-10-06T01:39:49.340791+0000","last_fullsized":"2023-10-06T01:39:49.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.886175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350835+0000","last_change":"2023-10-06T01:39:45.336408+0000","last_active":"2023-10-06T01:39:49.350835+0000","last_peered":"2023-10-06T01:39:49.350835+0000","last_clean":"2023-10-06T01:39:49.350835+0000","last_became_active":"2023-10-06T01:39:45.336066+0000","last_became_peered":"2023-10-06T01:39:45.336066+0000","last_unstale":"2023-10-06T01:39:49.350835+0000","last_undegraded":"2023-10-06T01:39:49.350835+0000","last_fullsized":"2023-10-06T01:39:49.350835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350056+0000","last_change":"2023-10-06T01:39:45.331003+0000","last_active":"2023-10-06T01:39:49.350056+0000","last_peered":"2023-10-06T01:39:49.350056+0000","last_clean":"2023-10-06T01:39:49.350056+0000","last_became_active":"2023-10-06T01:39:45.330563+0000","last_became_peered":"2023-10-06T01:39:45.330563+0000","last_unstale":"2023-10-06T01:39:49.350056+0000","last_undegraded":"2023-10-06T01:39:49.350056+0000","last_fullsized":"2023-10-06T01:39:49.350056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351292+0000","last_change":"2023-10-06T01:39:45.344173+0000","last_active":"2023-10-06T01:39:49.351292+0000","last_peered":"2023-10-06T01:39:49.351292+0000","last_clean":"2023-10-06T01:39:49.351292+0000","last_became_active":"2023-10-06T01:39:45.343891+0000","last_became_peered":"2023-10-06T01:39:45.343891+0000","last_unstale":"2023-10-06T01:39:49.351292+0000","last_undegraded":"2023-10-06T01:39:49.351292+0000","last_fullsized":"2023-10-06T01:39:49.351292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353439+0000","last_change":"2023-10-06T01:39:45.332773+0000","last_active":"2023-10-06T01:39:49.353439+0000","last_peered":"2023-10-06T01:39:49.353439+0000","last_clean":"2023-10-06T01:39:49.353439+0000","last_became_active":"2023-10-06T01:39:45.332504+0000","last_became_peered":"2023-10-06T01:39:45.332504+0000","last_unstale":"2023-10-06T01:39:49.353439+0000","last_undegraded":"2023-10-06T01:39:49.353439+0000","last_fullsized":"2023-10-06T01:39:49.353439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340177+0000","last_change":"2023-10-06T01:39:45.335528+0000","last_active":"2023-10-06T01:39:49.340177+0000","last_peered":"2023-10-06T01:39:49.340177+0000","last_clean":"2023-10-06T01:39:49.340177+0000","last_became_active":"2023-10-06T01:39:45.335296+0000","last_became_peered":"2023-10-06T01:39:45.335296+0000","last_unstale":"2023-10-06T01:39:49.340177+0000","last_undegraded":"2023-10-06T01:39:49.340177+0000","last_fullsized":"2023-10-06T01:39:49.340177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:09.004117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346177+0000","last_change":"2023-10-06T01:39:45.326855+0000","last_active":"2023-10-06T01:39:49.346177+0000","last_peered":"2023-10-06T01:39:49.346177+0000","last_clean":"2023-10-06T01:39:49.346177+0000","last_became_active":"2023-10-06T01:39:45.326260+0000","last_became_peered":"2023-10-06T01:39:45.326260+0000","last_unstale":"2023-10-06T01:39:49.346177+0000","last_undegraded":"2023-10-06T01:39:49.346177+0000","last_fullsized":"2023-10-06T01:39:49.346177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:26.407500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345996+0000","last_change":"2023-10-06T01:39:45.346287+0000","last_active":"2023-10-06T01:39:49.345996+0000","last_peered":"2023-10-06T01:39:49.345996+0000","last_clean":"2023-10-06T01:39:49.345996+0000","last_became_active":"2023-10-06T01:39:45.346017+0000","last_became_peered":"2023-10-06T01:39:45.346017+0000","last_unstale":"2023-10-06T01:39:49.345996+0000","last_undegraded":"2023-10-06T01:39:49.345996+0000","last_fullsized":"2023-10-06T01:39:49.345996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:37.090919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350016+0000","last_change":"2023-10-06T01:39:45.337191+0000","last_active":"2023-10-06T01:39:49.350016+0000","last_peered":"2023-10-06T01:39:49.350016+0000","last_clean":"2023-10-06T01:39:49.350016+0000","last_became_active":"2023-10-06T01:39:45.336905+0000","last_became_peered":"2023-10-06T01:39:45.336905+0000","last_unstale":"2023-10-06T01:39:49.350016+0000","last_undegraded":"2023-10-06T01:39:49.350016+0000","last_fullsized":"2023-10-06T01:39:49.350016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350613+0000","last_change":"2023-10-06T01:39:45.336653+0000","last_active":"2023-10-06T01:39:49.350613+0000","last_peered":"2023-10-06T01:39:49.350613+0000","last_clean":"2023-10-06T01:39:49.350613+0000","last_became_active":"2023-10-06T01:39:45.336383+0000","last_became_peered":"2023-10-06T01:39:45.336383+0000","last_unstale":"2023-10-06T01:39:49.350613+0000","last_undegraded":"2023-10-06T01:39:49.350613+0000","last_fullsized":"2023-10-06T01:39:49.350613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342395+0000","last_change":"2023-10-06T01:39:45.332271+0000","last_active":"2023-10-06T01:39:49.342395+0000","last_peered":"2023-10-06T01:39:49.342395+0000","last_clean":"2023-10-06T01:39:49.342395+0000","last_became_active":"2023-10-06T01:39:45.331844+0000","last_became_peered":"2023-10-06T01:39:45.331844+0000","last_unstale":"2023-10-06T01:39:49.342395+0000","last_undegraded":"2023-10-06T01:39:49.342395+0000","last_fullsized":"2023-10-06T01:39:49.342395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:01.791676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351382+0000","last_change":"2023-10-06T01:39:45.336079+0000","last_active":"2023-10-06T01:39:49.351382+0000","last_peered":"2023-10-06T01:39:49.351382+0000","last_clean":"2023-10-06T01:39:49.351382+0000","last_became_active":"2023-10-06T01:39:45.335828+0000","last_became_peered":"2023-10-06T01:39:45.335828+0000","last_unstale":"2023-10-06T01:39:49.351382+0000","last_undegraded":"2023-10-06T01:39:49.351382+0000","last_fullsized":"2023-10-06T01:39:49.351382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342168+0000","last_change":"2023-10-06T01:39:45.339626+0000","last_active":"2023-10-06T01:39:49.342168+0000","last_peered":"2023-10-06T01:39:49.342168+0000","last_clean":"2023-10-06T01:39:49.342168+0000","last_became_active":"2023-10-06T01:39:45.339370+0000","last_became_peered":"2023-10-06T01:39:45.339370+0000","last_unstale":"2023-10-06T01:39:49.342168+0000","last_undegraded":"2023-10-06T01:39:49.342168+0000","last_fullsized":"2023-10-06T01:39:49.342168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:12.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346086+0000","last_change":"2023-10-06T01:39:45.325821+0000","last_active":"2023-10-06T01:39:49.346086+0000","last_peered":"2023-10-06T01:39:49.346086+0000","last_clean":"2023-10-06T01:39:49.346086+0000","last_became_active":"2023-10-06T01:39:45.324033+0000","last_became_peered":"2023-10-06T01:39:45.324033+0000","last_unstale":"2023-10-06T01:39:49.346086+0000","last_undegraded":"2023-10-06T01:39:49.346086+0000","last_fullsized":"2023-10-06T01:39:49.346086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:13.558462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:45.330675+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:45.330406+0000","last_became_peered":"2023-10-06T01:39:45.330406+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350748+0000","last_change":"2023-10-06T01:39:45.342284+0000","last_active":"2023-10-06T01:39:49.350748+0000","last_peered":"2023-10-06T01:39:49.350748+0000","last_clean":"2023-10-06T01:39:49.350748+0000","last_became_active":"2023-10-06T01:39:45.342034+0000","last_became_peered":"2023-10-06T01:39:45.342034+0000","last_unstale":"2023-10-06T01:39:49.350748+0000","last_undegraded":"2023-10-06T01:39:49.350748+0000","last_fullsized":"2023-10-06T01:39:49.350748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347974+0000","last_change":"2023-10-06T01:39:45.334697+0000","last_active":"2023-10-06T01:39:49.347974+0000","last_peered":"2023-10-06T01:39:49.347974+0000","last_clean":"2023-10-06T01:39:49.347974+0000","last_became_active":"2023-10-06T01:39:45.334398+0000","last_became_peered":"2023-10-06T01:39:45.334398+0000","last_unstale":"2023-10-06T01:39:49.347974+0000","last_undegraded":"2023-10-06T01:39:49.347974+0000","last_fullsized":"2023-10-06T01:39:49.347974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338438+0000","last_change":"2023-10-06T01:39:45.336938+0000","last_active":"2023-10-06T01:39:49.338438+0000","last_peered":"2023-10-06T01:39:49.338438+0000","last_clean":"2023-10-06T01:39:49.338438+0000","last_became_active":"2023-10-06T01:39:45.336679+0000","last_became_peered":"2023-10-06T01:39:45.336679+0000","last_unstale":"2023-10-06T01:39:49.338438+0000","last_undegraded":"2023-10-06T01:39:49.338438+0000","last_fullsized":"2023-10-06T01:39:49.338438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350674+0000","last_change":"2023-10-06T01:39:45.338035+0000","last_active":"2023-10-06T01:39:49.350674+0000","last_peered":"2023-10-06T01:39:49.350674+0000","last_clean":"2023-10-06T01:39:49.350674+0000","last_became_active":"2023-10-06T01:39:45.337835+0000","last_became_peered":"2023-10-06T01:39:45.337835+0000","last_unstale":"2023-10-06T01:39:49.350674+0000","last_undegraded":"2023-10-06T01:39:49.350674+0000","last_fullsized":"2023-10-06T01:39:49.350674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344202+0000","last_change":"2023-10-06T01:39:45.328764+0000","last_active":"2023-10-06T01:39:49.344202+0000","last_peered":"2023-10-06T01:39:49.344202+0000","last_clean":"2023-10-06T01:39:49.344202+0000","last_became_active":"2023-10-06T01:39:45.328221+0000","last_became_peered":"2023-10-06T01:39:45.328221+0000","last_unstale":"2023-10-06T01:39:49.344202+0000","last_undegraded":"2023-10-06T01:39:49.344202+0000","last_fullsized":"2023-10-06T01:39:49.344202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344090+0000","last_change":"2023-10-06T01:39:45.337344+0000","last_active":"2023-10-06T01:39:49.344090+0000","last_peered":"2023-10-06T01:39:49.344090+0000","last_clean":"2023-10-06T01:39:49.344090+0000","last_became_active":"2023-10-06T01:39:45.336952+0000","last_became_peered":"2023-10-06T01:39:45.336952+0000","last_unstale":"2023-10-06T01:39:49.344090+0000","last_undegraded":"2023-10-06T01:39:49.344090+0000","last_fullsized":"2023-10-06T01:39:49.344090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350190+0000","last_change":"2023-10-06T01:39:45.334194+0000","last_active":"2023-10-06T01:39:49.350190+0000","last_peered":"2023-10-06T01:39:49.350190+0000","last_clean":"2023-10-06T01:39:49.350190+0000","last_became_active":"2023-10-06T01:39:45.333745+0000","last_became_peered":"2023-10-06T01:39:45.333745+0000","last_unstale":"2023-10-06T01:39:49.350190+0000","last_undegraded":"2023-10-06T01:39:49.350190+0000","last_fullsized":"2023-10-06T01:39:49.350190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360851+0000","last_change":"2023-10-06T01:39:49.359145+0000","last_active":"2023-10-06T01:39:49.360851+0000","last_peered":"2023-10-06T01:39:49.360851+0000","last_clean":"2023-10-06T01:39:49.360851+0000","last_became_active":"2023-10-06T01:39:49.358785+0000","last_became_peered":"2023-10-06T01:39:49.358785+0000","last_unstale":"2023-10-06T01:39:49.360851+0000","last_undegraded":"2023-10-06T01:39:49.360851+0000","last_fullsized":"2023-10-06T01:39:49.360851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:22.255025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340582+0000","last_change":"2023-10-06T01:39:44.325257+0000","last_active":"2023-10-06T01:39:49.340582+0000","last_peered":"2023-10-06T01:39:49.340582+0000","last_clean":"2023-10-06T01:39:49.340582+0000","last_became_active":"2023-10-06T01:39:44.324956+0000","last_became_peered":"2023-10-06T01:39:44.324956+0000","last_unstale":"2023-10-06T01:39:49.340582+0000","last_undegraded":"2023-10-06T01:39:49.340582+0000","last_fullsized":"2023-10-06T01:39:49.340582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338359+0000","last_change":"2023-10-06T01:39:45.331219+0000","last_active":"2023-10-06T01:39:49.338359+0000","last_peered":"2023-10-06T01:39:49.338359+0000","last_clean":"2023-10-06T01:39:49.338359+0000","last_became_active":"2023-10-06T01:39:45.330964+0000","last_became_peered":"2023-10-06T01:39:45.330964+0000","last_unstale":"2023-10-06T01:39:49.338359+0000","last_undegraded":"2023-10-06T01:39:49.338359+0000","last_fullsized":"2023-10-06T01:39:49.338359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359624+0000","last_change":"2023-10-06T01:39:49.356248+0000","last_active":"2023-10-06T01:39:49.359624+0000","last_peered":"2023-10-06T01:39:49.359624+0000","last_clean":"2023-10-06T01:39:49.359624+0000","last_became_active":"2023-10-06T01:39:49.355851+0000","last_became_peered":"2023-10-06T01:39:49.355851+0000","last_unstale":"2023-10-06T01:39:49.359624+0000","last_undegraded":"2023-10-06T01:39:49.359624+0000","last_fullsized":"2023-10-06T01:39:49.359624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341956+0000","last_change":"2023-10-06T01:39:44.327847+0000","last_active":"2023-10-06T01:39:49.341956+0000","last_peered":"2023-10-06T01:39:49.341956+0000","last_clean":"2023-10-06T01:39:49.341956+0000","last_became_active":"2023-10-06T01:39:44.327533+0000","last_became_peered":"2023-10-06T01:39:44.327533+0000","last_unstale":"2023-10-06T01:39:49.341956+0000","last_undegraded":"2023-10-06T01:39:49.341956+0000","last_fullsized":"2023-10-06T01:39:49.341956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351266+0000","last_change":"2023-10-06T01:39:45.326239+0000","last_active":"2023-10-06T01:39:49.351266+0000","last_peered":"2023-10-06T01:39:49.351266+0000","last_clean":"2023-10-06T01:39:49.351266+0000","last_became_active":"2023-10-06T01:39:45.325981+0000","last_became_peered":"2023-10-06T01:39:45.325981+0000","last_unstale":"2023-10-06T01:39:49.351266+0000","last_undegraded":"2023-10-06T01:39:49.351266+0000","last_fullsized":"2023-10-06T01:39:49.351266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361578+0000","last_change":"2023-10-06T01:39:49.358502+0000","last_active":"2023-10-06T01:39:49.361578+0000","last_peered":"2023-10-06T01:39:49.361578+0000","last_clean":"2023-10-06T01:39:49.361578+0000","last_became_active":"2023-10-06T01:39:49.358114+0000","last_became_peered":"2023-10-06T01:39:49.358114+0000","last_unstale":"2023-10-06T01:39:49.361578+0000","last_undegraded":"2023-10-06T01:39:49.361578+0000","last_fullsized":"2023-10-06T01:39:49.361578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338202+0000","last_change":"2023-10-06T01:39:44.325114+0000","last_active":"2023-10-06T01:39:49.338202+0000","last_peered":"2023-10-06T01:39:49.338202+0000","last_clean":"2023-10-06T01:39:49.338202+0000","last_became_active":"2023-10-06T01:39:44.324802+0000","last_became_peered":"2023-10-06T01:39:44.324802+0000","last_unstale":"2023-10-06T01:39:49.338202+0000","last_undegraded":"2023-10-06T01:39:49.338202+0000","last_fullsized":"2023-10-06T01:39:49.338202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:25.402016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339917+0000","last_change":"2023-10-06T01:39:45.337997+0000","last_active":"2023-10-06T01:39:49.339917+0000","last_peered":"2023-10-06T01:39:49.339917+0000","last_clean":"2023-10-06T01:39:49.339917+0000","last_became_active":"2023-10-06T01:39:45.337770+0000","last_became_peered":"2023-10-06T01:39:45.337770+0000","last_unstale":"2023-10-06T01:39:49.339917+0000","last_undegraded":"2023-10-06T01:39:49.339917+0000","last_fullsized":"2023-10-06T01:39:49.339917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359080+0000","last_change":"2023-10-06T01:39:49.355683+0000","last_active":"2023-10-06T01:39:49.359080+0000","last_peered":"2023-10-06T01:39:49.359080+0000","last_clean":"2023-10-06T01:39:49.359080+0000","last_became_active":"2023-10-06T01:39:49.355122+0000","last_became_peered":"2023-10-06T01:39:49.355122+0000","last_unstale":"2023-10-06T01:39:49.359080+0000","last_undegraded":"2023-10-06T01:39:49.359080+0000","last_fullsized":"2023-10-06T01:39:49.359080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351467+0000","last_change":"2023-10-06T01:39:44.326264+0000","last_active":"2023-10-06T01:39:49.351467+0000","last_peered":"2023-10-06T01:39:49.351467+0000","last_clean":"2023-10-06T01:39:49.351467+0000","last_became_active":"2023-10-06T01:39:44.325998+0000","last_became_peered":"2023-10-06T01:39:44.325998+0000","last_unstale":"2023-10-06T01:39:49.351467+0000","last_undegraded":"2023-10-06T01:39:49.351467+0000","last_fullsized":"2023-10-06T01:39:49.351467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338117+0000","last_change":"2023-10-06T01:39:45.332320+0000","last_active":"2023-10-06T01:39:49.338117+0000","last_peered":"2023-10-06T01:39:49.338117+0000","last_clean":"2023-10-06T01:39:49.338117+0000","last_became_active":"2023-10-06T01:39:45.332049+0000","last_became_peered":"2023-10-06T01:39:45.332049+0000","last_unstale":"2023-10-06T01:39:49.338117+0000","last_undegraded":"2023-10-06T01:39:49.338117+0000","last_fullsized":"2023-10-06T01:39:49.338117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363738+0000","last_change":"2023-10-06T01:39:49.360295+0000","last_active":"2023-10-06T01:39:49.363738+0000","last_peered":"2023-10-06T01:39:49.363738+0000","last_clean":"2023-10-06T01:39:49.363738+0000","last_became_active":"2023-10-06T01:39:49.359873+0000","last_became_peered":"2023-10-06T01:39:49.359873+0000","last_unstale":"2023-10-06T01:39:49.363738+0000","last_undegraded":"2023-10-06T01:39:49.363738+0000","last_fullsized":"2023-10-06T01:39:49.363738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356499+0000","last_change":"2023-10-06T01:39:44.328151+0000","last_active":"2023-10-06T01:39:49.356499+0000","last_peered":"2023-10-06T01:39:49.356499+0000","last_clean":"2023-10-06T01:39:49.356499+0000","last_became_active":"2023-10-06T01:39:44.327876+0000","last_became_peered":"2023-10-06T01:39:44.327876+0000","last_unstale":"2023-10-06T01:39:49.356499+0000","last_undegraded":"2023-10-06T01:39:49.356499+0000","last_fullsized":"2023-10-06T01:39:49.356499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337479+0000","last_change":"2023-10-06T01:39:45.330616+0000","last_active":"2023-10-06T01:39:49.337479+0000","last_peered":"2023-10-06T01:39:49.337479+0000","last_clean":"2023-10-06T01:39:49.337479+0000","last_became_active":"2023-10-06T01:39:45.330340+0000","last_became_peered":"2023-10-06T01:39:45.330340+0000","last_unstale":"2023-10-06T01:39:49.337479+0000","last_undegraded":"2023-10-06T01:39:49.337479+0000","last_fullsized":"2023-10-06T01:39:49.337479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368948+0000","last_change":"2023-10-06T01:39:49.365881+0000","last_active":"2023-10-06T01:39:49.368948+0000","last_peered":"2023-10-06T01:39:49.368948+0000","last_clean":"2023-10-06T01:39:49.368948+0000","last_became_active":"2023-10-06T01:39:49.365499+0000","last_became_peered":"2023-10-06T01:39:49.365499+0000","last_unstale":"2023-10-06T01:39:49.368948+0000","last_undegraded":"2023-10-06T01:39:49.368948+0000","last_fullsized":"2023-10-06T01:39:49.368948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:06.882659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351163+0000","last_change":"2023-10-06T01:39:44.326937+0000","last_active":"2023-10-06T01:39:49.351163+0000","last_peered":"2023-10-06T01:39:49.351163+0000","last_clean":"2023-10-06T01:39:49.351163+0000","last_became_active":"2023-10-06T01:39:44.326701+0000","last_became_peered":"2023-10-06T01:39:44.326701+0000","last_unstale":"2023-10-06T01:39:49.351163+0000","last_undegraded":"2023-10-06T01:39:49.351163+0000","last_fullsized":"2023-10-06T01:39:49.351163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:23.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342612+0000","last_change":"2023-10-06T01:39:45.331176+0000","last_active":"2023-10-06T01:39:49.342612+0000","last_peered":"2023-10-06T01:39:49.342612+0000","last_clean":"2023-10-06T01:39:49.342612+0000","last_became_active":"2023-10-06T01:39:45.330927+0000","last_became_peered":"2023-10-06T01:39:45.330927+0000","last_unstale":"2023-10-06T01:39:49.342612+0000","last_undegraded":"2023-10-06T01:39:49.342612+0000","last_fullsized":"2023-10-06T01:39:49.342612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357166+0000","last_change":"2023-10-06T01:39:49.355332+0000","last_active":"2023-10-06T01:39:49.357166+0000","last_peered":"2023-10-06T01:39:49.357166+0000","last_clean":"2023-10-06T01:39:49.357166+0000","last_became_active":"2023-10-06T01:39:49.354961+0000","last_became_peered":"2023-10-06T01:39:49.354961+0000","last_unstale":"2023-10-06T01:39:49.357166+0000","last_undegraded":"2023-10-06T01:39:49.357166+0000","last_fullsized":"2023-10-06T01:39:49.357166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T01:39:55.162081+0000","last_change":"2023-10-06T01:39:44.325901+0000","last_active":"2023-10-06T01:39:55.162081+0000","last_peered":"2023-10-06T01:39:55.162081+0000","last_clean":"2023-10-06T01:39:55.162081+0000","last_became_active":"2023-10-06T01:39:44.325631+0000","last_became_peered":"2023-10-06T01:39:44.325631+0000","last_unstale":"2023-10-06T01:39:55.162081+0000","last_undegraded":"2023-10-06T01:39:55.162081+0000","last_fullsized":"2023-10-06T01:39:55.162081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340321+0000","last_change":"2023-10-06T01:39:45.328644+0000","last_active":"2023-10-06T01:39:49.340321+0000","last_peered":"2023-10-06T01:39:49.340321+0000","last_clean":"2023-10-06T01:39:49.340321+0000","last_became_active":"2023-10-06T01:39:45.328367+0000","last_became_peered":"2023-10-06T01:39:45.328367+0000","last_unstale":"2023-10-06T01:39:49.340321+0000","last_undegraded":"2023-10-06T01:39:49.340321+0000","last_fullsized":"2023-10-06T01:39:49.340321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367347+0000","last_change":"2023-10-06T01:39:49.364574+0000","last_active":"2023-10-06T01:39:49.367347+0000","last_peered":"2023-10-06T01:39:49.367347+0000","last_clean":"2023-10-06T01:39:49.367347+0000","last_became_active":"2023-10-06T01:39:49.360749+0000","last_became_peered":"2023-10-06T01:39:49.360749+0000","last_unstale":"2023-10-06T01:39:49.367347+0000","last_undegraded":"2023-10-06T01:39:49.367347+0000","last_fullsized":"2023-10-06T01:39:49.367347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350570+0000","last_change":"2023-10-06T01:39:44.332362+0000","last_active":"2023-10-06T01:39:49.350570+0000","last_peered":"2023-10-06T01:39:49.350570+0000","last_clean":"2023-10-06T01:39:49.350570+0000","last_became_active":"2023-10-06T01:39:44.332082+0000","last_became_peered":"2023-10-06T01:39:44.332082+0000","last_unstale":"2023-10-06T01:39:49.350570+0000","last_undegraded":"2023-10-06T01:39:49.350570+0000","last_fullsized":"2023-10-06T01:39:49.350570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338444+0000","last_change":"2023-10-06T01:39:45.333142+0000","last_active":"2023-10-06T01:39:49.338444+0000","last_peered":"2023-10-06T01:39:49.338444+0000","last_clean":"2023-10-06T01:39:49.338444+0000","last_became_active":"2023-10-06T01:39:45.332880+0000","last_became_peered":"2023-10-06T01:39:45.332880+0000","last_unstale":"2023-10-06T01:39:49.338444+0000","last_undegraded":"2023-10-06T01:39:49.338444+0000","last_fullsized":"2023-10-06T01:39:49.338444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359341+0000","last_change":"2023-10-06T01:39:49.357412+0000","last_active":"2023-10-06T01:39:49.359341+0000","last_peered":"2023-10-06T01:39:49.359341+0000","last_clean":"2023-10-06T01:39:49.359341+0000","last_became_active":"2023-10-06T01:39:49.357031+0000","last_became_peered":"2023-10-06T01:39:49.357031+0000","last_unstale":"2023-10-06T01:39:49.359341+0000","last_undegraded":"2023-10-06T01:39:49.359341+0000","last_fullsized":"2023-10-06T01:39:49.359341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.348119+0000","last_change":"2023-10-06T01:39:44.325262+0000","last_active":"2023-10-06T01:39:49.348119+0000","last_peered":"2023-10-06T01:39:49.348119+0000","last_clean":"2023-10-06T01:39:49.348119+0000","last_became_active":"2023-10-06T01:39:44.325017+0000","last_became_peered":"2023-10-06T01:39:44.325017+0000","last_unstale":"2023-10-06T01:39:49.348119+0000","last_undegraded":"2023-10-06T01:39:49.348119+0000","last_fullsized":"2023-10-06T01:39:49.348119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350108+0000","last_change":"2023-10-06T01:39:45.334022+0000","last_active":"2023-10-06T01:39:49.350108+0000","last_peered":"2023-10-06T01:39:49.350108+0000","last_clean":"2023-10-06T01:39:49.350108+0000","last_became_active":"2023-10-06T01:39:45.333462+0000","last_became_peered":"2023-10-06T01:39:45.333462+0000","last_unstale":"2023-10-06T01:39:49.350108+0000","last_undegraded":"2023-10-06T01:39:49.350108+0000","last_fullsized":"2023-10-06T01:39:49.350108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.358648+0000","last_change":"2023-10-06T01:39:49.357484+0000","last_active":"2023-10-06T01:39:49.358648+0000","last_peered":"2023-10-06T01:39:49.358648+0000","last_clean":"2023-10-06T01:39:49.358648+0000","last_became_active":"2023-10-06T01:39:49.356839+0000","last_became_peered":"2023-10-06T01:39:49.356839+0000","last_unstale":"2023-10-06T01:39:49.358648+0000","last_undegraded":"2023-10-06T01:39:49.358648+0000","last_fullsized":"2023-10-06T01:39:49.358648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:14.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350742+0000","last_change":"2023-10-06T01:39:44.333226+0000","last_active":"2023-10-06T01:39:49.350742+0000","last_peered":"2023-10-06T01:39:49.350742+0000","last_clean":"2023-10-06T01:39:49.350742+0000","last_became_active":"2023-10-06T01:39:44.332920+0000","last_became_peered":"2023-10-06T01:39:44.332920+0000","last_unstale":"2023-10-06T01:39:49.350742+0000","last_undegraded":"2023-10-06T01:39:49.350742+0000","last_fullsized":"2023-10-06T01:39:49.350742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.344914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351529+0000","last_change":"2023-10-06T01:39:45.333240+0000","last_active":"2023-10-06T01:39:49.351529+0000","last_peered":"2023-10-06T01:39:49.351529+0000","last_clean":"2023-10-06T01:39:49.351529+0000","last_became_active":"2023-10-06T01:39:45.332985+0000","last_became_peered":"2023-10-06T01:39:45.332985+0000","last_unstale":"2023-10-06T01:39:49.351529+0000","last_undegraded":"2023-10-06T01:39:49.351529+0000","last_fullsized":"2023-10-06T01:39:49.351529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.364965+0000","last_change":"2023-10-06T01:39:49.360274+0000","last_active":"2023-10-06T01:39:49.364965+0000","last_peered":"2023-10-06T01:39:49.364965+0000","last_clean":"2023-10-06T01:39:49.364965+0000","last_became_active":"2023-10-06T01:39:49.359287+0000","last_became_peered":"2023-10-06T01:39:49.359287+0000","last_unstale":"2023-10-06T01:39:49.364965+0000","last_undegraded":"2023-10-06T01:39:49.364965+0000","last_fullsized":"2023-10-06T01:39:49.364965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:39.297464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343143+0000","last_change":"2023-10-06T01:39:44.322253+0000","last_active":"2023-10-06T01:39:49.343143+0000","last_peered":"2023-10-06T01:39:49.343143+0000","last_clean":"2023-10-06T01:39:49.343143+0000","last_became_active":"2023-10-06T01:39:44.321953+0000","last_became_peered":"2023-10-06T01:39:44.321953+0000","last_unstale":"2023-10-06T01:39:49.343143+0000","last_undegraded":"2023-10-06T01:39:49.343143+0000","last_fullsized":"2023-10-06T01:39:49.343143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338146+0000","last_change":"2023-10-06T01:39:45.343656+0000","last_active":"2023-10-06T01:39:49.338146+0000","last_peered":"2023-10-06T01:39:49.338146+0000","last_clean":"2023-10-06T01:39:49.338146+0000","last_became_active":"2023-10-06T01:39:45.342592+0000","last_became_peered":"2023-10-06T01:39:45.342592+0000","last_unstale":"2023-10-06T01:39:49.338146+0000","last_undegraded":"2023-10-06T01:39:49.338146+0000","last_fullsized":"2023-10-06T01:39:49.338146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365064+0000","last_change":"2023-10-06T01:39:49.363708+0000","last_active":"2023-10-06T01:39:49.365064+0000","last_peered":"2023-10-06T01:39:49.365064+0000","last_clean":"2023-10-06T01:39:49.365064+0000","last_became_active":"2023-10-06T01:39:49.363055+0000","last_became_peered":"2023-10-06T01:39:49.363055+0000","last_unstale":"2023-10-06T01:39:49.365064+0000","last_undegraded":"2023-10-06T01:39:49.365064+0000","last_fullsized":"2023-10-06T01:39:49.365064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352442+0000","last_change":"2023-10-06T01:39:44.333358+0000","last_active":"2023-10-06T01:39:49.352442+0000","last_peered":"2023-10-06T01:39:49.352442+0000","last_clean":"2023-10-06T01:39:49.352442+0000","last_became_active":"2023-10-06T01:39:44.332979+0000","last_became_peered":"2023-10-06T01:39:44.332979+0000","last_unstale":"2023-10-06T01:39:49.352442+0000","last_undegraded":"2023-10-06T01:39:49.352442+0000","last_fullsized":"2023-10-06T01:39:49.352442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349928+0000","last_change":"2023-10-06T01:39:45.330806+0000","last_active":"2023-10-06T01:39:49.349928+0000","last_peered":"2023-10-06T01:39:49.349928+0000","last_clean":"2023-10-06T01:39:49.349928+0000","last_became_active":"2023-10-06T01:39:45.330280+0000","last_became_peered":"2023-10-06T01:39:45.330280+0000","last_unstale":"2023-10-06T01:39:49.349928+0000","last_undegraded":"2023-10-06T01:39:49.349928+0000","last_fullsized":"2023-10-06T01:39:49.349928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.355843+0000","last_change":"2023-10-06T01:39:49.352785+0000","last_active":"2023-10-06T01:39:49.355843+0000","last_peered":"2023-10-06T01:39:49.355843+0000","last_clean":"2023-10-06T01:39:49.355843+0000","last_became_active":"2023-10-06T01:39:49.352396+0000","last_became_peered":"2023-10-06T01:39:49.352396+0000","last_unstale":"2023-10-06T01:39:49.355843+0000","last_undegraded":"2023-10-06T01:39:49.355843+0000","last_fullsized":"2023-10-06T01:39:49.355843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:45.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352263+0000","last_change":"2023-10-06T01:39:44.332277+0000","last_active":"2023-10-06T01:39:49.352263+0000","last_peered":"2023-10-06T01:39:49.352263+0000","last_clean":"2023-10-06T01:39:49.352263+0000","last_became_active":"2023-10-06T01:39:44.331987+0000","last_became_peered":"2023-10-06T01:39:44.331987+0000","last_unstale":"2023-10-06T01:39:49.352263+0000","last_undegraded":"2023-10-06T01:39:49.352263+0000","last_fullsized":"2023-10-06T01:39:49.352263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:36.012185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342030+0000","last_change":"2023-10-06T01:39:45.326893+0000","last_active":"2023-10-06T01:39:49.342030+0000","last_peered":"2023-10-06T01:39:49.342030+0000","last_clean":"2023-10-06T01:39:49.342030+0000","last_became_active":"2023-10-06T01:39:45.326610+0000","last_became_peered":"2023-10-06T01:39:45.326610+0000","last_unstale":"2023-10-06T01:39:49.342030+0000","last_undegraded":"2023-10-06T01:39:49.342030+0000","last_fullsized":"2023-10-06T01:39:49.342030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360816+0000","last_change":"2023-10-06T01:39:49.359772+0000","last_active":"2023-10-06T01:39:49.360816+0000","last_peered":"2023-10-06T01:39:49.360816+0000","last_clean":"2023-10-06T01:39:49.360816+0000","last_became_active":"2023-10-06T01:39:49.359414+0000","last_became_peered":"2023-10-06T01:39:49.359414+0000","last_unstale":"2023-10-06T01:39:49.360816+0000","last_undegraded":"2023-10-06T01:39:49.360816+0000","last_fullsized":"2023-10-06T01:39:49.360816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351061+0000","last_change":"2023-10-06T01:39:44.330033+0000","last_active":"2023-10-06T01:39:49.351061+0000","last_peered":"2023-10-06T01:39:49.351061+0000","last_clean":"2023-10-06T01:39:49.351061+0000","last_became_active":"2023-10-06T01:39:44.329732+0000","last_became_peered":"2023-10-06T01:39:44.329732+0000","last_unstale":"2023-10-06T01:39:49.351061+0000","last_undegraded":"2023-10-06T01:39:49.351061+0000","last_fullsized":"2023-10-06T01:39:49.351061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339868+0000","last_change":"2023-10-06T01:39:45.334185+0000","last_active":"2023-10-06T01:39:49.339868+0000","last_peered":"2023-10-06T01:39:49.339868+0000","last_clean":"2023-10-06T01:39:49.339868+0000","last_became_active":"2023-10-06T01:39:45.333053+0000","last_became_peered":"2023-10-06T01:39:45.333053+0000","last_unstale":"2023-10-06T01:39:49.339868+0000","last_undegraded":"2023-10-06T01:39:49.339868+0000","last_fullsized":"2023-10-06T01:39:49.339868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:58.429853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356192+0000","last_change":"2023-10-06T01:39:49.353291+0000","last_active":"2023-10-06T01:39:49.356192+0000","last_peered":"2023-10-06T01:39:49.356192+0000","last_clean":"2023-10-06T01:39:49.356192+0000","last_became_active":"2023-10-06T01:39:49.352912+0000","last_became_peered":"2023-10-06T01:39:49.352912+0000","last_unstale":"2023-10-06T01:39:49.356192+0000","last_undegraded":"2023-10-06T01:39:49.356192+0000","last_fullsized":"2023-10-06T01:39:49.356192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.229209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.325148+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.324687+0000","last_became_peered":"2023-10-06T01:39:44.324687+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347753+0000","last_change":"2023-10-06T01:39:45.329116+0000","last_active":"2023-10-06T01:39:49.347753+0000","last_peered":"2023-10-06T01:39:49.347753+0000","last_clean":"2023-10-06T01:39:49.347753+0000","last_became_active":"2023-10-06T01:39:45.328839+0000","last_became_peered":"2023-10-06T01:39:45.328839+0000","last_unstale":"2023-10-06T01:39:49.347753+0000","last_undegraded":"2023-10-06T01:39:49.347753+0000","last_fullsized":"2023-10-06T01:39:49.347753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356181+0000","last_change":"2023-10-06T01:39:49.353796+0000","last_active":"2023-10-06T01:39:49.356181+0000","last_peered":"2023-10-06T01:39:49.356181+0000","last_clean":"2023-10-06T01:39:49.356181+0000","last_became_active":"2023-10-06T01:39:49.353138+0000","last_became_peered":"2023-10-06T01:39:49.353138+0000","last_unstale":"2023-10-06T01:39:49.356181+0000","last_undegraded":"2023-10-06T01:39:49.356181+0000","last_fullsized":"2023-10-06T01:39:49.356181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337716+0000","last_change":"2023-10-06T01:39:44.329032+0000","last_active":"2023-10-06T01:39:49.337716+0000","last_peered":"2023-10-06T01:39:49.337716+0000","last_clean":"2023-10-06T01:39:49.337716+0000","last_became_active":"2023-10-06T01:39:44.328775+0000","last_became_peered":"2023-10-06T01:39:44.328775+0000","last_unstale":"2023-10-06T01:39:49.337716+0000","last_undegraded":"2023-10-06T01:39:49.337716+0000","last_fullsized":"2023-10-06T01:39:49.337716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331469+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.331152+0000","last_became_peered":"2023-10-06T01:39:45.331152+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356783+0000","last_change":"2023-10-06T01:39:49.351619+0000","last_active":"2023-10-06T01:39:49.356783+0000","last_peered":"2023-10-06T01:39:49.356783+0000","last_clean":"2023-10-06T01:39:49.356783+0000","last_became_active":"2023-10-06T01:39:49.351271+0000","last_became_peered":"2023-10-06T01:39:49.351271+0000","last_unstale":"2023-10-06T01:39:49.356783+0000","last_undegraded":"2023-10-06T01:39:49.356783+0000","last_fullsized":"2023-10-06T01:39:49.356783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:05.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339967+0000","last_change":"2023-10-06T01:39:44.321209+0000","last_active":"2023-10-06T01:39:49.339967+0000","last_peered":"2023-10-06T01:39:49.339967+0000","last_clean":"2023-10-06T01:39:49.339967+0000","last_became_active":"2023-10-06T01:39:44.320932+0000","last_became_peered":"2023-10-06T01:39:44.320932+0000","last_unstale":"2023-10-06T01:39:49.339967+0000","last_undegraded":"2023-10-06T01:39:49.339967+0000","last_fullsized":"2023-10-06T01:39:49.339967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339741+0000","last_change":"2023-10-06T01:39:45.339859+0000","last_active":"2023-10-06T01:39:49.339741+0000","last_peered":"2023-10-06T01:39:49.339741+0000","last_clean":"2023-10-06T01:39:49.339741+0000","last_became_active":"2023-10-06T01:39:45.339422+0000","last_became_peered":"2023-10-06T01:39:45.339422+0000","last_unstale":"2023-10-06T01:39:49.339741+0000","last_undegraded":"2023-10-06T01:39:49.339741+0000","last_fullsized":"2023-10-06T01:39:49.339741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:36.266399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367304+0000","last_change":"2023-10-06T01:39:49.361654+0000","last_active":"2023-10-06T01:39:49.367304+0000","last_peered":"2023-10-06T01:39:49.367304+0000","last_clean":"2023-10-06T01:39:49.367304+0000","last_became_active":"2023-10-06T01:39:49.361257+0000","last_became_peered":"2023-10-06T01:39:49.361257+0000","last_unstale":"2023-10-06T01:39:49.367304+0000","last_undegraded":"2023-10-06T01:39:49.367304+0000","last_fullsized":"2023-10-06T01:39:49.367304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:13:40.277782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341922+0000","last_change":"2023-10-06T01:39:44.326380+0000","last_active":"2023-10-06T01:39:49.341922+0000","last_peered":"2023-10-06T01:39:49.341922+0000","last_clean":"2023-10-06T01:39:49.341922+0000","last_became_active":"2023-10-06T01:39:44.326059+0000","last_became_peered":"2023-10-06T01:39:44.326059+0000","last_unstale":"2023-10-06T01:39:49.341922+0000","last_undegraded":"2023-10-06T01:39:49.341922+0000","last_fullsized":"2023-10-06T01:39:49.341922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342079+0000","last_change":"2023-10-06T01:39:45.329138+0000","last_active":"2023-10-06T01:39:49.342079+0000","last_peered":"2023-10-06T01:39:49.342079+0000","last_clean":"2023-10-06T01:39:49.342079+0000","last_became_active":"2023-10-06T01:39:45.328861+0000","last_became_peered":"2023-10-06T01:39:45.328861+0000","last_unstale":"2023-10-06T01:39:49.342079+0000","last_undegraded":"2023-10-06T01:39:49.342079+0000","last_fullsized":"2023-10-06T01:39:49.342079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365004+0000","last_change":"2023-10-06T01:39:49.360752+0000","last_active":"2023-10-06T01:39:49.365004+0000","last_peered":"2023-10-06T01:39:49.365004+0000","last_clean":"2023-10-06T01:39:49.365004+0000","last_became_active":"2023-10-06T01:39:49.360376+0000","last_became_peered":"2023-10-06T01:39:49.360376+0000","last_unstale":"2023-10-06T01:39:49.365004+0000","last_undegraded":"2023-10-06T01:39:49.365004+0000","last_fullsized":"2023-10-06T01:39:49.365004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.092106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338022+0000","last_change":"2023-10-06T01:39:44.325482+0000","last_active":"2023-10-06T01:39:49.338022+0000","last_peered":"2023-10-06T01:39:49.338022+0000","last_clean":"2023-10-06T01:39:49.338022+0000","last_became_active":"2023-10-06T01:39:44.325160+0000","last_became_peered":"2023-10-06T01:39:44.325160+0000","last_unstale":"2023-10-06T01:39:49.338022+0000","last_undegraded":"2023-10-06T01:39:49.338022+0000","last_fullsized":"2023-10-06T01:39:49.338022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344360+0000","last_change":"2023-10-06T01:39:45.342331+0000","last_active":"2023-10-06T01:39:49.344360+0000","last_peered":"2023-10-06T01:39:49.344360+0000","last_clean":"2023-10-06T01:39:49.344360+0000","last_became_active":"2023-10-06T01:39:45.342022+0000","last_became_peered":"2023-10-06T01:39:45.342022+0000","last_unstale":"2023-10-06T01:39:49.344360+0000","last_undegraded":"2023-10-06T01:39:49.344360+0000","last_fullsized":"2023-10-06T01:39:49.344360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357018+0000","last_change":"2023-10-06T01:39:49.355423+0000","last_active":"2023-10-06T01:39:49.357018+0000","last_peered":"2023-10-06T01:39:49.357018+0000","last_clean":"2023-10-06T01:39:49.357018+0000","last_became_active":"2023-10-06T01:39:49.355075+0000","last_became_peered":"2023-10-06T01:39:49.355075+0000","last_unstale":"2023-10-06T01:39:49.357018+0000","last_undegraded":"2023-10-06T01:39:49.357018+0000","last_fullsized":"2023-10-06T01:39:49.357018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:02.539610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338012+0000","last_change":"2023-10-06T01:39:44.328632+0000","last_active":"2023-10-06T01:39:49.338012+0000","last_peered":"2023-10-06T01:39:49.338012+0000","last_clean":"2023-10-06T01:39:49.338012+0000","last_became_active":"2023-10-06T01:39:44.326328+0000","last_became_peered":"2023-10-06T01:39:44.326328+0000","last_unstale":"2023-10-06T01:39:49.338012+0000","last_undegraded":"2023-10-06T01:39:49.338012+0000","last_fullsized":"2023-10-06T01:39:49.338012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339913+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:39:49.339913+0000","last_peered":"2023-10-06T01:39:49.339913+0000","last_clean":"2023-10-06T01:39:49.339913+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:39:49.339913+0000","last_undegraded":"2023-10-06T01:39:49.339913+0000","last_fullsized":"2023-10-06T01:39:49.339913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337712+0000","last_change":"2023-10-06T01:39:45.342039+0000","last_active":"2023-10-06T01:39:49.337712+0000","last_peered":"2023-10-06T01:39:49.337712+0000","last_clean":"2023-10-06T01:39:49.337712+0000","last_became_active":"2023-10-06T01:39:45.341746+0000","last_became_peered":"2023-10-06T01:39:45.341746+0000","last_unstale":"2023-10-06T01:39:49.337712+0000","last_undegraded":"2023-10-06T01:39:49.337712+0000","last_fullsized":"2023-10-06T01:39:49.337712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354087+0000","last_change":"2023-10-06T01:39:49.352264+0000","last_active":"2023-10-06T01:39:49.354087+0000","last_peered":"2023-10-06T01:39:49.354087+0000","last_clean":"2023-10-06T01:39:49.354087+0000","last_became_active":"2023-10-06T01:39:49.351974+0000","last_became_peered":"2023-10-06T01:39:49.351974+0000","last_unstale":"2023-10-06T01:39:49.354087+0000","last_undegraded":"2023-10-06T01:39:49.354087+0000","last_fullsized":"2023-10-06T01:39:49.354087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:35.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337764+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:39:49.337764+0000","last_peered":"2023-10-06T01:39:49.337764+0000","last_clean":"2023-10-06T01:39:49.337764+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:39:49.337764+0000","last_undegraded":"2023-10-06T01:39:49.337764+0000","last_fullsized":"2023-10-06T01:39:49.337764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.395905+0000","last_change":"2023-10-06T01:39:44.324833+0000","last_active":"2023-10-06T01:39:49.395905+0000","last_peered":"2023-10-06T01:39:49.395905+0000","last_clean":"2023-10-06T01:39:49.395905+0000","last_became_active":"2023-10-06T01:39:44.324554+0000","last_became_peered":"2023-10-06T01:39:44.324554+0000","last_unstale":"2023-10-06T01:39:49.395905+0000","last_undegraded":"2023-10-06T01:39:49.395905+0000","last_fullsized":"2023-10-06T01:39:49.395905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:53.581079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341933+0000","last_change":"2023-10-06T01:39:45.331099+0000","last_active":"2023-10-06T01:39:49.341933+0000","last_peered":"2023-10-06T01:39:49.341933+0000","last_clean":"2023-10-06T01:39:49.341933+0000","last_became_active":"2023-10-06T01:39:45.330844+0000","last_became_peered":"2023-10-06T01:39:45.330844+0000","last_unstale":"2023-10-06T01:39:49.341933+0000","last_undegraded":"2023-10-06T01:39:49.341933+0000","last_fullsized":"2023-10-06T01:39:49.341933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367282+0000","last_change":"2023-10-06T01:39:49.360839+0000","last_active":"2023-10-06T01:39:49.367282+0000","last_peered":"2023-10-06T01:39:49.367282+0000","last_clean":"2023-10-06T01:39:49.367282+0000","last_became_active":"2023-10-06T01:39:49.360447+0000","last_became_peered":"2023-10-06T01:39:49.360447+0000","last_unstale":"2023-10-06T01:39:49.367282+0000","last_undegraded":"2023-10-06T01:39:49.367282+0000","last_fullsized":"2023-10-06T01:39:49.367282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342223+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:39:49.342223+0000","last_peered":"2023-10-06T01:39:49.342223+0000","last_clean":"2023-10-06T01:39:49.342223+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:39:49.342223+0000","last_undegraded":"2023-10-06T01:39:49.342223+0000","last_fullsized":"2023-10-06T01:39:49.342223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350783+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:39:49.350783+0000","last_peered":"2023-10-06T01:39:49.350783+0000","last_clean":"2023-10-06T01:39:49.350783+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:39:49.350783+0000","last_undegraded":"2023-10-06T01:39:49.350783+0000","last_fullsized":"2023-10-06T01:39:49.350783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346294+0000","last_change":"2023-10-06T01:39:44.332942+0000","last_active":"2023-10-06T01:39:49.346294+0000","last_peered":"2023-10-06T01:39:49.346294+0000","last_clean":"2023-10-06T01:39:49.346294+0000","last_became_active":"2023-10-06T01:39:44.332767+0000","last_became_peered":"2023-10-06T01:39:44.332767+0000","last_unstale":"2023-10-06T01:39:49.346294+0000","last_undegraded":"2023-10-06T01:39:49.346294+0000","last_fullsized":"2023-10-06T01:39:49.346294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351451+0000","last_change":"2023-10-06T01:39:45.331974+0000","last_active":"2023-10-06T01:39:49.351451+0000","last_peered":"2023-10-06T01:39:49.351451+0000","last_clean":"2023-10-06T01:39:49.351451+0000","last_became_active":"2023-10-06T01:39:45.331706+0000","last_became_peered":"2023-10-06T01:39:45.331706+0000","last_unstale":"2023-10-06T01:39:49.351451+0000","last_undegraded":"2023-10-06T01:39:49.351451+0000","last_fullsized":"2023-10-06T01:39:49.351451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357328+0000","last_change":"2023-10-06T01:39:49.356514+0000","last_active":"2023-10-06T01:39:49.357328+0000","last_peered":"2023-10-06T01:39:49.357328+0000","last_clean":"2023-10-06T01:39:49.357328+0000","last_became_active":"2023-10-06T01:39:49.356017+0000","last_became_peered":"2023-10-06T01:39:49.356017+0000","last_unstale":"2023-10-06T01:39:49.357328+0000","last_undegraded":"2023-10-06T01:39:49.357328+0000","last_fullsized":"2023-10-06T01:39:49.357328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:28.616739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350977+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:39:49.350977+0000","last_peered":"2023-10-06T01:39:49.350977+0000","last_clean":"2023-10-06T01:39:49.350977+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:39:49.350977+0000","last_undegraded":"2023-10-06T01:39:49.350977+0000","last_fullsized":"2023-10-06T01:39:49.350977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345374+0000","last_change":"2023-10-06T01:39:44.324538+0000","last_active":"2023-10-06T01:39:49.345374+0000","last_peered":"2023-10-06T01:39:49.345374+0000","last_clean":"2023-10-06T01:39:49.345374+0000","last_became_active":"2023-10-06T01:39:44.324239+0000","last_became_peered":"2023-10-06T01:39:44.324239+0000","last_unstale":"2023-10-06T01:39:49.345374+0000","last_undegraded":"2023-10-06T01:39:49.345374+0000","last_fullsized":"2023-10-06T01:39:49.345374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344282+0000","last_change":"2023-10-06T01:39:45.339432+0000","last_active":"2023-10-06T01:39:49.344282+0000","last_peered":"2023-10-06T01:39:49.344282+0000","last_clean":"2023-10-06T01:39:49.344282+0000","last_became_active":"2023-10-06T01:39:45.339191+0000","last_became_peered":"2023-10-06T01:39:45.339191+0000","last_unstale":"2023-10-06T01:39:49.344282+0000","last_undegraded":"2023-10-06T01:39:49.344282+0000","last_fullsized":"2023-10-06T01:39:49.344282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363746+0000","last_change":"2023-10-06T01:39:49.361536+0000","last_active":"2023-10-06T01:39:49.363746+0000","last_peered":"2023-10-06T01:39:49.363746+0000","last_clean":"2023-10-06T01:39:49.363746+0000","last_became_active":"2023-10-06T01:39:49.361162+0000","last_became_peered":"2023-10-06T01:39:49.361162+0000","last_unstale":"2023-10-06T01:39:49.363746+0000","last_undegraded":"2023-10-06T01:39:49.363746+0000","last_fullsized":"2023-10-06T01:39:49.363746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350576+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:39:49.350576+0000","last_peered":"2023-10-06T01:39:49.350576+0000","last_clean":"2023-10-06T01:39:49.350576+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:39:49.350576+0000","last_undegraded":"2023-10-06T01:39:49.350576+0000","last_fullsized":"2023-10-06T01:39:49.350576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337530+0000","last_change":"2023-10-06T01:39:44.328264+0000","last_active":"2023-10-06T01:39:49.337530+0000","last_peered":"2023-10-06T01:39:49.337530+0000","last_clean":"2023-10-06T01:39:49.337530+0000","last_became_active":"2023-10-06T01:39:44.327969+0000","last_became_peered":"2023-10-06T01:39:44.327969+0000","last_unstale":"2023-10-06T01:39:49.337530+0000","last_undegraded":"2023-10-06T01:39:49.337530+0000","last_fullsized":"2023-10-06T01:39:49.337530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340324+0000","last_change":"2023-10-06T01:39:45.333753+0000","last_active":"2023-10-06T01:39:49.340324+0000","last_peered":"2023-10-06T01:39:49.340324+0000","last_clean":"2023-10-06T01:39:49.340324+0000","last_became_active":"2023-10-06T01:39:45.333416+0000","last_became_peered":"2023-10-06T01:39:45.333416+0000","last_unstale":"2023-10-06T01:39:49.340324+0000","last_undegraded":"2023-10-06T01:39:49.340324+0000","last_fullsized":"2023-10-06T01:39:49.340324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361707+0000","last_change":"2023-10-06T01:39:49.360037+0000","last_active":"2023-10-06T01:39:49.361707+0000","last_peered":"2023-10-06T01:39:49.361707+0000","last_clean":"2023-10-06T01:39:49.361707+0000","last_became_active":"2023-10-06T01:39:49.359697+0000","last_became_peered":"2023-10-06T01:39:49.359697+0000","last_unstale":"2023-10-06T01:39:49.361707+0000","last_undegraded":"2023-10-06T01:39:49.361707+0000","last_fullsized":"2023-10-06T01:39:49.361707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349757+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:39:49.349757+0000","last_peered":"2023-10-06T01:39:49.349757+0000","last_clean":"2023-10-06T01:39:49.349757+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:39:49.349757+0000","last_undegraded":"2023-10-06T01:39:49.349757+0000","last_fullsized":"2023-10-06T01:39:49.349757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:44.329991+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:44.329756+0000","last_became_peered":"2023-10-06T01:39:44.329756+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343619+0000","last_change":"2023-10-06T01:39:45.336684+0000","last_active":"2023-10-06T01:39:49.343619+0000","last_peered":"2023-10-06T01:39:49.343619+0000","last_clean":"2023-10-06T01:39:49.343619+0000","last_became_active":"2023-10-06T01:39:45.336427+0000","last_became_peered":"2023-10-06T01:39:45.336427+0000","last_unstale":"2023-10-06T01:39:49.343619+0000","last_undegraded":"2023-10-06T01:39:49.343619+0000","last_fullsized":"2023-10-06T01:39:49.343619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359866+0000","last_change":"2023-10-06T01:39:49.358138+0000","last_active":"2023-10-06T01:39:49.359866+0000","last_peered":"2023-10-06T01:39:49.359866+0000","last_clean":"2023-10-06T01:39:49.359866+0000","last_became_active":"2023-10-06T01:39:49.356144+0000","last_became_peered":"2023-10-06T01:39:49.356144+0000","last_unstale":"2023-10-06T01:39:49.359866+0000","last_undegraded":"2023-10-06T01:39:49.359866+0000","last_fullsized":"2023-10-06T01:39:49.359866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:02.622952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341234+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:39:49.341234+0000","last_peered":"2023-10-06T01:39:49.341234+0000","last_clean":"2023-10-06T01:39:49.341234+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:39:49.341234+0000","last_undegraded":"2023-10-06T01:39:49.341234+0000","last_fullsized":"2023-10-06T01:39:49.341234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343734+0000","last_change":"2023-10-06T01:39:44.327413+0000","last_active":"2023-10-06T01:39:49.343734+0000","last_peered":"2023-10-06T01:39:49.343734+0000","last_clean":"2023-10-06T01:39:49.343734+0000","last_became_active":"2023-10-06T01:39:44.327177+0000","last_became_peered":"2023-10-06T01:39:44.327177+0000","last_unstale":"2023-10-06T01:39:49.343734+0000","last_undegraded":"2023-10-06T01:39:49.343734+0000","last_fullsized":"2023-10-06T01:39:49.343734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:17.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347454+0000","last_change":"2023-10-06T01:39:45.334858+0000","last_active":"2023-10-06T01:39:49.347454+0000","last_peered":"2023-10-06T01:39:49.347454+0000","last_clean":"2023-10-06T01:39:49.347454+0000","last_became_active":"2023-10-06T01:39:45.334586+0000","last_became_peered":"2023-10-06T01:39:45.334586+0000","last_unstale":"2023-10-06T01:39:49.347454+0000","last_undegraded":"2023-10-06T01:39:49.347454+0000","last_fullsized":"2023-10-06T01:39:49.347454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:10.350398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.366854+0000","last_change":"2023-10-06T01:39:49.365711+0000","last_active":"2023-10-06T01:39:49.366854+0000","last_peered":"2023-10-06T01:39:49.366854+0000","last_clean":"2023-10-06T01:39:49.366854+0000","last_became_active":"2023-10-06T01:39:49.365357+0000","last_became_peered":"2023-10-06T01:39:49.365357+0000","last_unstale":"2023-10-06T01:39:49.366854+0000","last_undegraded":"2023-10-06T01:39:49.366854+0000","last_fullsized":"2023-10-06T01:39:49.366854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339829+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:39:49.339829+0000","last_peered":"2023-10-06T01:39:49.339829+0000","last_clean":"2023-10-06T01:39:49.339829+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:39:49.339829+0000","last_undegraded":"2023-10-06T01:39:49.339829+0000","last_fullsized":"2023-10-06T01:39:49.339829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350343+0000","last_change":"2023-10-06T01:39:44.327795+0000","last_active":"2023-10-06T01:39:49.350343+0000","last_peered":"2023-10-06T01:39:49.350343+0000","last_clean":"2023-10-06T01:39:49.350343+0000","last_became_active":"2023-10-06T01:39:44.326791+0000","last_became_peered":"2023-10-06T01:39:44.326791+0000","last_unstale":"2023-10-06T01:39:49.350343+0000","last_undegraded":"2023-10-06T01:39:49.350343+0000","last_fullsized":"2023-10-06T01:39:49.350343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342538+0000","last_change":"2023-10-06T01:39:45.341418+0000","last_active":"2023-10-06T01:39:49.342538+0000","last_peered":"2023-10-06T01:39:49.342538+0000","last_clean":"2023-10-06T01:39:49.342538+0000","last_became_active":"2023-10-06T01:39:45.341053+0000","last_became_peered":"2023-10-06T01:39:45.341053+0000","last_unstale":"2023-10-06T01:39:49.342538+0000","last_undegraded":"2023-10-06T01:39:49.342538+0000","last_fullsized":"2023-10-06T01:39:49.342538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356900+0000","last_change":"2023-10-06T01:39:49.355119+0000","last_active":"2023-10-06T01:39:49.356900+0000","last_peered":"2023-10-06T01:39:49.356900+0000","last_clean":"2023-10-06T01:39:49.356900+0000","last_became_active":"2023-10-06T01:39:49.354770+0000","last_became_peered":"2023-10-06T01:39:49.354770+0000","last_unstale":"2023-10-06T01:39:49.356900+0000","last_undegraded":"2023-10-06T01:39:49.356900+0000","last_fullsized":"2023-10-06T01:39:49.356900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:40.295194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347370+0000","last_change":"2023-10-06T01:39:44.333929+0000","last_active":"2023-10-06T01:39:49.347370+0000","last_peered":"2023-10-06T01:39:49.347370+0000","last_clean":"2023-10-06T01:39:49.347370+0000","last_became_active":"2023-10-06T01:39:44.333659+0000","last_became_peered":"2023-10-06T01:39:44.333659+0000","last_unstale":"2023-10-06T01:39:49.347370+0000","last_undegraded":"2023-10-06T01:39:49.347370+0000","last_fullsized":"2023-10-06T01:39:49.347370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346317+0000","last_change":"2023-10-06T01:39:45.331228+0000","last_active":"2023-10-06T01:39:49.346317+0000","last_peered":"2023-10-06T01:39:49.346317+0000","last_clean":"2023-10-06T01:39:49.346317+0000","last_became_active":"2023-10-06T01:39:45.330982+0000","last_became_peered":"2023-10-06T01:39:45.330982+0000","last_unstale":"2023-10-06T01:39:49.346317+0000","last_undegraded":"2023-10-06T01:39:49.346317+0000","last_fullsized":"2023-10-06T01:39:49.346317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.923135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363774+0000","last_change":"2023-10-06T01:39:49.362572+0000","last_active":"2023-10-06T01:39:49.363774+0000","last_peered":"2023-10-06T01:39:49.363774+0000","last_clean":"2023-10-06T01:39:49.363774+0000","last_became_active":"2023-10-06T01:39:49.362200+0000","last_became_peered":"2023-10-06T01:39:49.362200+0000","last_unstale":"2023-10-06T01:39:49.363774+0000","last_undegraded":"2023-10-06T01:39:49.363774+0000","last_fullsized":"2023-10-06T01:39:49.363774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349796+0000","last_change":"2023-10-06T01:39:44.332451+0000","last_active":"2023-10-06T01:39:49.349796+0000","last_peered":"2023-10-06T01:39:49.349796+0000","last_clean":"2023-10-06T01:39:49.349796+0000","last_became_active":"2023-10-06T01:39:44.332143+0000","last_became_peered":"2023-10-06T01:39:44.332143+0000","last_unstale":"2023-10-06T01:39:49.349796+0000","last_undegraded":"2023-10-06T01:39:49.349796+0000","last_fullsized":"2023-10-06T01:39:49.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331260+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.330980+0000","last_became_peered":"2023-10-06T01:39:45.330980+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368960+0000","last_change":"2023-10-06T01:39:49.361125+0000","last_active":"2023-10-06T01:39:49.368960+0000","last_peered":"2023-10-06T01:39:49.368960+0000","last_clean":"2023-10-06T01:39:49.368960+0000","last_became_active":"2023-10-06T01:39:49.354967+0000","last_became_peered":"2023-10-06T01:39:49.354967+0000","last_unstale":"2023-10-06T01:39:49.368960+0000","last_undegraded":"2023-10-06T01:39:49.368960+0000","last_fullsized":"2023-10-06T01:39:49.368960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:41.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.329353+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.329098+0000","last_became_peered":"2023-10-06T01:39:44.329098+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340491+0000","last_change":"2023-10-06T01:39:45.330780+0000","last_active":"2023-10-06T01:39:49.340491+0000","last_peered":"2023-10-06T01:39:49.340491+0000","last_clean":"2023-10-06T01:39:49.340491+0000","last_became_active":"2023-10-06T01:39:45.330484+0000","last_became_peered":"2023-10-06T01:39:45.330484+0000","last_unstale":"2023-10-06T01:39:49.340491+0000","last_undegraded":"2023-10-06T01:39:49.340491+0000","last_fullsized":"2023-10-06T01:39:49.340491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354659+0000","last_change":"2023-10-06T01:39:49.351706+0000","last_active":"2023-10-06T01:39:49.354659+0000","last_peered":"2023-10-06T01:39:49.354659+0000","last_clean":"2023-10-06T01:39:49.354659+0000","last_became_active":"2023-10-06T01:39:49.350630+0000","last_became_peered":"2023-10-06T01:39:49.350630+0000","last_unstale":"2023-10-06T01:39:49.354659+0000","last_undegraded":"2023-10-06T01:39:49.354659+0000","last_fullsized":"2023-10-06T01:39:49.354659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:57.336637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337644+0000","last_change":"2023-10-06T01:39:44.321387+0000","last_active":"2023-10-06T01:39:49.337644+0000","last_peered":"2023-10-06T01:39:49.337644+0000","last_clean":"2023-10-06T01:39:49.337644+0000","last_became_active":"2023-10-06T01:39:44.321101+0000","last_became_peered":"2023-10-06T01:39:44.321101+0000","last_unstale":"2023-10-06T01:39:49.337644+0000","last_undegraded":"2023-10-06T01:39:49.337644+0000","last_fullsized":"2023-10-06T01:39:49.337644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337678+0000","last_change":"2023-10-06T01:39:45.333689+0000","last_active":"2023-10-06T01:39:49.337678+0000","last_peered":"2023-10-06T01:39:49.337678+0000","last_clean":"2023-10-06T01:39:49.337678+0000","last_became_active":"2023-10-06T01:39:45.333444+0000","last_became_peered":"2023-10-06T01:39:45.333444+0000","last_unstale":"2023-10-06T01:39:49.337678+0000","last_undegraded":"2023-10-06T01:39:49.337678+0000","last_fullsized":"2023-10-06T01:39:49.337678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.362749+0000","last_change":"2023-10-06T01:39:49.360875+0000","last_active":"2023-10-06T01:39:49.362749+0000","last_peered":"2023-10-06T01:39:49.362749+0000","last_clean":"2023-10-06T01:39:49.362749+0000","last_became_active":"2023-10-06T01:39:49.360515+0000","last_became_peered":"2023-10-06T01:39:49.360515+0000","last_unstale":"2023-10-06T01:39:49.362749+0000","last_undegraded":"2023-10-06T01:39:49.362749+0000","last_fullsized":"2023-10-06T01:39:49.362749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351554+0000","last_change":"2023-10-06T01:39:44.326433+0000","last_active":"2023-10-06T01:39:49.351554+0000","last_peered":"2023-10-06T01:39:49.351554+0000","last_clean":"2023-10-06T01:39:49.351554+0000","last_became_active":"2023-10-06T01:39:44.326167+0000","last_became_peered":"2023-10-06T01:39:44.326167+0000","last_unstale":"2023-10-06T01:39:49.351554+0000","last_undegraded":"2023-10-06T01:39:49.351554+0000","last_fullsized":"2023-10-06T01:39:49.351554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342356+0000","last_change":"2023-10-06T01:39:45.330018+0000","last_active":"2023-10-06T01:39:49.342356+0000","last_peered":"2023-10-06T01:39:49.342356+0000","last_clean":"2023-10-06T01:39:49.342356+0000","last_became_active":"2023-10-06T01:39:45.329556+0000","last_became_peered":"2023-10-06T01:39:45.329556+0000","last_unstale":"2023-10-06T01:39:49.342356+0000","last_undegraded":"2023-10-06T01:39:49.342356+0000","last_fullsized":"2023-10-06T01:39:49.342356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343235+0000","last_change":"2023-10-06T01:39:45.333252+0000","last_active":"2023-10-06T01:39:49.343235+0000","last_peered":"2023-10-06T01:39:49.343235+0000","last_clean":"2023-10-06T01:39:49.343235+0000","last_became_active":"2023-10-06T01:39:45.332899+0000","last_became_peered":"2023-10-06T01:39:45.332899+0000","last_unstale":"2023-10-06T01:39:49.343235+0000","last_undegraded":"2023-10-06T01:39:49.343235+0000","last_fullsized":"2023-10-06T01:39:49.343235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:37.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337491+0000","last_change":"2023-10-06T01:39:45.342104+0000","last_active":"2023-10-06T01:39:49.337491+0000","last_peered":"2023-10-06T01:39:49.337491+0000","last_clean":"2023-10-06T01:39:49.337491+0000","last_became_active":"2023-10-06T01:39:45.341825+0000","last_became_peered":"2023-10-06T01:39:45.341825+0000","last_unstale":"2023-10-06T01:39:49.337491+0000","last_undegraded":"2023-10-06T01:39:49.337491+0000","last_fullsized":"2023-10-06T01:39:49.337491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340694+0000","last_change":"2023-10-06T01:39:45.329243+0000","last_active":"2023-10-06T01:39:49.340694+0000","last_peered":"2023-10-06T01:39:49.340694+0000","last_clean":"2023-10-06T01:39:49.340694+0000","last_became_active":"2023-10-06T01:39:45.328438+0000","last_became_peered":"2023-10-06T01:39:45.328438+0000","last_unstale":"2023-10-06T01:39:49.340694+0000","last_undegraded":"2023-10-06T01:39:49.340694+0000","last_fullsized":"2023-10-06T01:39:49.340694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337590+0000","last_change":"2023-10-06T01:39:45.330664+0000","last_active":"2023-10-06T01:39:49.337590+0000","last_peered":"2023-10-06T01:39:49.337590+0000","last_clean":"2023-10-06T01:39:49.337590+0000","last_became_active":"2023-10-06T01:39:45.330394+0000","last_became_peered":"2023-10-06T01:39:45.330394+0000","last_unstale":"2023-10-06T01:39:49.337590+0000","last_undegraded":"2023-10-06T01:39:49.337590+0000","last_fullsized":"2023-10-06T01:39:49.337590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343700+0000","last_change":"2023-10-06T01:39:45.338933+0000","last_active":"2023-10-06T01:39:49.343700+0000","last_peered":"2023-10-06T01:39:49.343700+0000","last_clean":"2023-10-06T01:39:49.343700+0000","last_became_active":"2023-10-06T01:39:45.338515+0000","last_became_peered":"2023-10-06T01:39:45.338515+0000","last_unstale":"2023-10-06T01:39:49.343700+0000","last_undegraded":"2023-10-06T01:39:49.343700+0000","last_fullsized":"2023-10-06T01:39:49.343700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6619999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8019999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.3090000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.452}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:03.578 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:03.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.726+0000 7f6967f44700 1 Processor -- start 2023-10-06T01:40:03.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.726+0000 7f6967f44700 1 -- start start 2023-10-06T01:40:03.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f6967f44700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6960159b10 0x7f6960159f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f6967f44700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f69600ca6b0 0x7f69600caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f6967f44700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69600cb010 con 0x7f69600cb630 2023-10-06T01:40:03.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f6967f44700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696015a4d0 con 0x7f69600ca6b0 2023-10-06T01:40:03.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f6967f44700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6960154060 con 0x7f6960159b10 2023-10-06T01:40:03.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f69664e1700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f69600ca6b0 0x7f69600caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f69664e1700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f69600ca6b0 0x7f69600caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41922/0 (socket says 172.21.15.98:41922) 2023-10-06T01:40:03.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.727+0000 7f69664e1700 1 -- 172.21.15.98:0/3349865195 learned_addr learned my addr 172.21.15.98:0/3349865195 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:03.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69654df700 1 --2- 172.21.15.98:0/3349865195 >> v2:172.21.15.184:3300/0 conn(0x7f6960159b10 0x7f6960159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69664e1700 1 -- 172.21.15.98:0/3349865195 >> v2:172.21.15.184:3300/0 conn(0x7f6960159b10 msgr2=0x7f6960159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:03.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69664e1700 1 --2- 172.21.15.98:0/3349865195 >> v2:172.21.15.184:3300/0 conn(0x7f6960159b10 0x7f6960159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69664e1700 1 -- 172.21.15.98:0/3349865195 >> v1:172.21.15.184:6789/0 conn(0x7f69600cb630 legacy=0x7f69600691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:03.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69664e1700 1 -- 172.21.15.98:0/3349865195 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960068d50 con 0x7f69600ca6b0 2023-10-06T01:40:03.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f69664e1700 1 --2- 172.21.15.98:0/3349865195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69600ca6b0 0x7f69600caad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f696006f300 tx=0x7f6958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295bedb17f353788 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6964cde700 1 -- 172.21.15.98:0/3349865195 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800b7e0 con 0x7f69600ca6b0 2023-10-06T01:40:03.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6964cde700 1 -- 172.21.15.98:0/3349865195 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695800b940 con 0x7f69600ca6b0 2023-10-06T01:40:03.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6964cde700 1 -- 172.21.15.98:0/3349865195 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800bce0 con 0x7f69600ca6b0 2023-10-06T01:40:03.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 -- 172.21.15.98:0/3349865195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69600ca6b0 msgr2=0x7f69600caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 --2- 172.21.15.98:0/3349865195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69600ca6b0 0x7f69600caad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f696006f300 tx=0x7f6958009250 comp rx=0 tx=0).stop 2023-10-06T01:40:03.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 -- 172.21.15.98:0/3349865195 shutdown_connections 2023-10-06T01:40:03.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 --2- 172.21.15.98:0/3349865195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69600ca6b0 0x7f69600caad0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 --2- 172.21.15.98:0/3349865195 >> v2:172.21.15.184:3300/0 conn(0x7f6960159b10 0x7f6960159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.728+0000 7f6967f44700 1 -- 172.21.15.98:0/3349865195 >> 172.21.15.98:0/3349865195 conn(0x7f6960070810 msgr2=0x7f6960070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.729+0000 7f6967f44700 1 -- 172.21.15.98:0/3349865195 shutdown_connections 2023-10-06T01:40:03.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.729+0000 7f6967f44700 1 -- 172.21.15.98:0/3349865195 wait complete. 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 Processor -- start 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 -- start start 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 0x7f69600b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 0x7f69601556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600b3e20 con 0x7f6960159b10 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600c77d0 con 0x7f69600cb630 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f6967f44700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6960155c20 con 0x7f69600ca6b0 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f69664e1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f69664e1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41938/0 (socket says 172.21.15.98:41938) 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f69664e1700 1 -- 172.21.15.98:0/3388284820 learned_addr learned my addr 172.21.15.98:0/3388284820 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.730+0000 7f69654df700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 0x7f69601556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6965ce0700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 0x7f69600b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 -- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 msgr2=0x7f69600b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 0x7f69600b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 -- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 msgr2=0x7f69601556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 0x7f69601556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 -- 172.21.15.98:0/3388284820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580094f0 con 0x7f6960159b10 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69654df700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 0x7f69601556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6965ce0700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 0x7f69600b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f69664e1700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6958009290 tx=0x7f6958016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695801c660 con 0x7f6960159b10 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695801c7e0 con 0x7f6960159b10 2023-10-06T01:40:03.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69600c6ea0 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695801cba0 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.731+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69600c7460 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.732+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696006c710 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.733+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f695802f480 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.733+0000 7f6955ffb700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.733+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6958086b00 con 0x7f6960159b10 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.733+0000 7f6965ce0700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.734+0000 7f6965ce0700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 0x7f69440693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f695c007fd0 tx=0x7f695c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:03.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.737+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6958053900 con 0x7f6960159b10 2023-10-06T01:40:03.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.917+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69600cb010 con 0x7f6944066f30 2023-10-06T01:40:03.928 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.927+0000 7f6955ffb700 1 -- 172.21.15.98:0/3388284820 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367616 (secure 0 0 0) 0x7f69600cb010 con 0x7f6944066f30 2023-10-06T01:40:03.933 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 msgr2=0x7f69440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 0x7f69440693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f695c007fd0 tx=0x7f695c009040 comp rx=0 tx=0).stop 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 msgr2=0x7f696016a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6958009290 tx=0x7f6958016000 comp rx=0 tx=0).stop 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 shutdown_connections 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6960159b10 0x7f696016a2b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69600ca6b0 0x7f69600b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 --2- 172.21.15.98:0/3388284820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69600cb630 0x7f69601556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:03.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 >> 172.21.15.98:0/3388284820 conn(0x7f6960070810 msgr2=0x7f6960153310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:03.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 shutdown_connections 2023-10-06T01:40:03.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:03.931+0000 7f6967f44700 1 -- 172.21.15.98:0/3388284820 wait complete. 2023-10-06T01:40:03.939 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:03.944 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-06T01:40:03.788796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002728"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346761+0000","last_change":"2023-10-06T01:39:45.335803+0000","last_active":"2023-10-06T01:39:49.346761+0000","last_peered":"2023-10-06T01:39:49.346761+0000","last_clean":"2023-10-06T01:39:49.346761+0000","last_became_active":"2023-10-06T01:39:45.335420+0000","last_became_peered":"2023-10-06T01:39:45.335420+0000","last_unstale":"2023-10-06T01:39:49.346761+0000","last_undegraded":"2023-10-06T01:39:49.346761+0000","last_fullsized":"2023-10-06T01:39:49.346761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:58.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353326+0000","last_change":"2023-10-06T01:39:45.336392+0000","last_active":"2023-10-06T01:39:49.353326+0000","last_peered":"2023-10-06T01:39:49.353326+0000","last_clean":"2023-10-06T01:39:49.353326+0000","last_became_active":"2023-10-06T01:39:45.336114+0000","last_became_peered":"2023-10-06T01:39:45.336114+0000","last_unstale":"2023-10-06T01:39:49.353326+0000","last_undegraded":"2023-10-06T01:39:49.353326+0000","last_fullsized":"2023-10-06T01:39:49.353326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340003+0000","last_change":"2023-10-06T01:39:45.330889+0000","last_active":"2023-10-06T01:39:49.340003+0000","last_peered":"2023-10-06T01:39:49.340003+0000","last_clean":"2023-10-06T01:39:49.340003+0000","last_became_active":"2023-10-06T01:39:45.330584+0000","last_became_peered":"2023-10-06T01:39:45.330584+0000","last_unstale":"2023-10-06T01:39:49.340003+0000","last_undegraded":"2023-10-06T01:39:49.340003+0000","last_fullsized":"2023-10-06T01:39:49.340003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:29:55.095137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340381+0000","last_change":"2023-10-06T01:39:45.327678+0000","last_active":"2023-10-06T01:39:49.340381+0000","last_peered":"2023-10-06T01:39:49.340381+0000","last_clean":"2023-10-06T01:39:49.340381+0000","last_became_active":"2023-10-06T01:39:45.327329+0000","last_became_peered":"2023-10-06T01:39:45.327329+0000","last_unstale":"2023-10-06T01:39:49.340381+0000","last_undegraded":"2023-10-06T01:39:49.340381+0000","last_fullsized":"2023-10-06T01:39:49.340381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:13.570152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347538+0000","last_change":"2023-10-06T01:39:45.342805+0000","last_active":"2023-10-06T01:39:49.347538+0000","last_peered":"2023-10-06T01:39:49.347538+0000","last_clean":"2023-10-06T01:39:49.347538+0000","last_became_active":"2023-10-06T01:39:45.342510+0000","last_became_peered":"2023-10-06T01:39:45.342510+0000","last_unstale":"2023-10-06T01:39:49.347538+0000","last_undegraded":"2023-10-06T01:39:49.347538+0000","last_fullsized":"2023-10-06T01:39:49.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340791+0000","last_change":"2023-10-06T01:39:45.328521+0000","last_active":"2023-10-06T01:39:49.340791+0000","last_peered":"2023-10-06T01:39:49.340791+0000","last_clean":"2023-10-06T01:39:49.340791+0000","last_became_active":"2023-10-06T01:39:45.327249+0000","last_became_peered":"2023-10-06T01:39:45.327249+0000","last_unstale":"2023-10-06T01:39:49.340791+0000","last_undegraded":"2023-10-06T01:39:49.340791+0000","last_fullsized":"2023-10-06T01:39:49.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.886175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350835+0000","last_change":"2023-10-06T01:39:45.336408+0000","last_active":"2023-10-06T01:39:49.350835+0000","last_peered":"2023-10-06T01:39:49.350835+0000","last_clean":"2023-10-06T01:39:49.350835+0000","last_became_active":"2023-10-06T01:39:45.336066+0000","last_became_peered":"2023-10-06T01:39:45.336066+0000","last_unstale":"2023-10-06T01:39:49.350835+0000","last_undegraded":"2023-10-06T01:39:49.350835+0000","last_fullsized":"2023-10-06T01:39:49.350835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350056+0000","last_change":"2023-10-06T01:39:45.331003+0000","last_active":"2023-10-06T01:39:49.350056+0000","last_peered":"2023-10-06T01:39:49.350056+0000","last_clean":"2023-10-06T01:39:49.350056+0000","last_became_active":"2023-10-06T01:39:45.330563+0000","last_became_peered":"2023-10-06T01:39:45.330563+0000","last_unstale":"2023-10-06T01:39:49.350056+0000","last_undegraded":"2023-10-06T01:39:49.350056+0000","last_fullsized":"2023-10-06T01:39:49.350056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351292+0000","last_change":"2023-10-06T01:39:45.344173+0000","last_active":"2023-10-06T01:39:49.351292+0000","last_peered":"2023-10-06T01:39:49.351292+0000","last_clean":"2023-10-06T01:39:49.351292+0000","last_became_active":"2023-10-06T01:39:45.343891+0000","last_became_peered":"2023-10-06T01:39:45.343891+0000","last_unstale":"2023-10-06T01:39:49.351292+0000","last_undegraded":"2023-10-06T01:39:49.351292+0000","last_fullsized":"2023-10-06T01:39:49.351292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.353439+0000","last_change":"2023-10-06T01:39:45.332773+0000","last_active":"2023-10-06T01:39:49.353439+0000","last_peered":"2023-10-06T01:39:49.353439+0000","last_clean":"2023-10-06T01:39:49.353439+0000","last_became_active":"2023-10-06T01:39:45.332504+0000","last_became_peered":"2023-10-06T01:39:45.332504+0000","last_unstale":"2023-10-06T01:39:49.353439+0000","last_undegraded":"2023-10-06T01:39:49.353439+0000","last_fullsized":"2023-10-06T01:39:49.353439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340177+0000","last_change":"2023-10-06T01:39:45.335528+0000","last_active":"2023-10-06T01:39:49.340177+0000","last_peered":"2023-10-06T01:39:49.340177+0000","last_clean":"2023-10-06T01:39:49.340177+0000","last_became_active":"2023-10-06T01:39:45.335296+0000","last_became_peered":"2023-10-06T01:39:45.335296+0000","last_unstale":"2023-10-06T01:39:49.340177+0000","last_undegraded":"2023-10-06T01:39:49.340177+0000","last_fullsized":"2023-10-06T01:39:49.340177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:09.004117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346177+0000","last_change":"2023-10-06T01:39:45.326855+0000","last_active":"2023-10-06T01:39:49.346177+0000","last_peered":"2023-10-06T01:39:49.346177+0000","last_clean":"2023-10-06T01:39:49.346177+0000","last_became_active":"2023-10-06T01:39:45.326260+0000","last_became_peered":"2023-10-06T01:39:45.326260+0000","last_unstale":"2023-10-06T01:39:49.346177+0000","last_undegraded":"2023-10-06T01:39:49.346177+0000","last_fullsized":"2023-10-06T01:39:49.346177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:26.407500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345996+0000","last_change":"2023-10-06T01:39:45.346287+0000","last_active":"2023-10-06T01:39:49.345996+0000","last_peered":"2023-10-06T01:39:49.345996+0000","last_clean":"2023-10-06T01:39:49.345996+0000","last_became_active":"2023-10-06T01:39:45.346017+0000","last_became_peered":"2023-10-06T01:39:45.346017+0000","last_unstale":"2023-10-06T01:39:49.345996+0000","last_undegraded":"2023-10-06T01:39:49.345996+0000","last_fullsized":"2023-10-06T01:39:49.345996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:37.090919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350016+0000","last_change":"2023-10-06T01:39:45.337191+0000","last_active":"2023-10-06T01:39:49.350016+0000","last_peered":"2023-10-06T01:39:49.350016+0000","last_clean":"2023-10-06T01:39:49.350016+0000","last_became_active":"2023-10-06T01:39:45.336905+0000","last_became_peered":"2023-10-06T01:39:45.336905+0000","last_unstale":"2023-10-06T01:39:49.350016+0000","last_undegraded":"2023-10-06T01:39:49.350016+0000","last_fullsized":"2023-10-06T01:39:49.350016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350613+0000","last_change":"2023-10-06T01:39:45.336653+0000","last_active":"2023-10-06T01:39:49.350613+0000","last_peered":"2023-10-06T01:39:49.350613+0000","last_clean":"2023-10-06T01:39:49.350613+0000","last_became_active":"2023-10-06T01:39:45.336383+0000","last_became_peered":"2023-10-06T01:39:45.336383+0000","last_unstale":"2023-10-06T01:39:49.350613+0000","last_undegraded":"2023-10-06T01:39:49.350613+0000","last_fullsized":"2023-10-06T01:39:49.350613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342395+0000","last_change":"2023-10-06T01:39:45.332271+0000","last_active":"2023-10-06T01:39:49.342395+0000","last_peered":"2023-10-06T01:39:49.342395+0000","last_clean":"2023-10-06T01:39:49.342395+0000","last_became_active":"2023-10-06T01:39:45.331844+0000","last_became_peered":"2023-10-06T01:39:45.331844+0000","last_unstale":"2023-10-06T01:39:49.342395+0000","last_undegraded":"2023-10-06T01:39:49.342395+0000","last_fullsized":"2023-10-06T01:39:49.342395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:01.791676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351382+0000","last_change":"2023-10-06T01:39:45.336079+0000","last_active":"2023-10-06T01:39:49.351382+0000","last_peered":"2023-10-06T01:39:49.351382+0000","last_clean":"2023-10-06T01:39:49.351382+0000","last_became_active":"2023-10-06T01:39:45.335828+0000","last_became_peered":"2023-10-06T01:39:45.335828+0000","last_unstale":"2023-10-06T01:39:49.351382+0000","last_undegraded":"2023-10-06T01:39:49.351382+0000","last_fullsized":"2023-10-06T01:39:49.351382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342168+0000","last_change":"2023-10-06T01:39:45.339626+0000","last_active":"2023-10-06T01:39:49.342168+0000","last_peered":"2023-10-06T01:39:49.342168+0000","last_clean":"2023-10-06T01:39:49.342168+0000","last_became_active":"2023-10-06T01:39:45.339370+0000","last_became_peered":"2023-10-06T01:39:45.339370+0000","last_unstale":"2023-10-06T01:39:49.342168+0000","last_undegraded":"2023-10-06T01:39:49.342168+0000","last_fullsized":"2023-10-06T01:39:49.342168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:12.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346086+0000","last_change":"2023-10-06T01:39:45.325821+0000","last_active":"2023-10-06T01:39:49.346086+0000","last_peered":"2023-10-06T01:39:49.346086+0000","last_clean":"2023-10-06T01:39:49.346086+0000","last_became_active":"2023-10-06T01:39:45.324033+0000","last_became_peered":"2023-10-06T01:39:45.324033+0000","last_unstale":"2023-10-06T01:39:49.346086+0000","last_undegraded":"2023-10-06T01:39:49.346086+0000","last_fullsized":"2023-10-06T01:39:49.346086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:13.558462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:45.330675+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:45.330406+0000","last_became_peered":"2023-10-06T01:39:45.330406+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350748+0000","last_change":"2023-10-06T01:39:45.342284+0000","last_active":"2023-10-06T01:39:49.350748+0000","last_peered":"2023-10-06T01:39:49.350748+0000","last_clean":"2023-10-06T01:39:49.350748+0000","last_became_active":"2023-10-06T01:39:45.342034+0000","last_became_peered":"2023-10-06T01:39:45.342034+0000","last_unstale":"2023-10-06T01:39:49.350748+0000","last_undegraded":"2023-10-06T01:39:49.350748+0000","last_fullsized":"2023-10-06T01:39:49.350748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347974+0000","last_change":"2023-10-06T01:39:45.334697+0000","last_active":"2023-10-06T01:39:49.347974+0000","last_peered":"2023-10-06T01:39:49.347974+0000","last_clean":"2023-10-06T01:39:49.347974+0000","last_became_active":"2023-10-06T01:39:45.334398+0000","last_became_peered":"2023-10-06T01:39:45.334398+0000","last_unstale":"2023-10-06T01:39:49.347974+0000","last_undegraded":"2023-10-06T01:39:49.347974+0000","last_fullsized":"2023-10-06T01:39:49.347974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338438+0000","last_change":"2023-10-06T01:39:45.336938+0000","last_active":"2023-10-06T01:39:49.338438+0000","last_peered":"2023-10-06T01:39:49.338438+0000","last_clean":"2023-10-06T01:39:49.338438+0000","last_became_active":"2023-10-06T01:39:45.336679+0000","last_became_peered":"2023-10-06T01:39:45.336679+0000","last_unstale":"2023-10-06T01:39:49.338438+0000","last_undegraded":"2023-10-06T01:39:49.338438+0000","last_fullsized":"2023-10-06T01:39:49.338438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350674+0000","last_change":"2023-10-06T01:39:45.338035+0000","last_active":"2023-10-06T01:39:49.350674+0000","last_peered":"2023-10-06T01:39:49.350674+0000","last_clean":"2023-10-06T01:39:49.350674+0000","last_became_active":"2023-10-06T01:39:45.337835+0000","last_became_peered":"2023-10-06T01:39:45.337835+0000","last_unstale":"2023-10-06T01:39:49.350674+0000","last_undegraded":"2023-10-06T01:39:49.350674+0000","last_fullsized":"2023-10-06T01:39:49.350674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344202+0000","last_change":"2023-10-06T01:39:45.328764+0000","last_active":"2023-10-06T01:39:49.344202+0000","last_peered":"2023-10-06T01:39:49.344202+0000","last_clean":"2023-10-06T01:39:49.344202+0000","last_became_active":"2023-10-06T01:39:45.328221+0000","last_became_peered":"2023-10-06T01:39:45.328221+0000","last_unstale":"2023-10-06T01:39:49.344202+0000","last_undegraded":"2023-10-06T01:39:49.344202+0000","last_fullsized":"2023-10-06T01:39:49.344202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344090+0000","last_change":"2023-10-06T01:39:45.337344+0000","last_active":"2023-10-06T01:39:49.344090+0000","last_peered":"2023-10-06T01:39:49.344090+0000","last_clean":"2023-10-06T01:39:49.344090+0000","last_became_active":"2023-10-06T01:39:45.336952+0000","last_became_peered":"2023-10-06T01:39:45.336952+0000","last_unstale":"2023-10-06T01:39:49.344090+0000","last_undegraded":"2023-10-06T01:39:49.344090+0000","last_fullsized":"2023-10-06T01:39:49.344090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350190+0000","last_change":"2023-10-06T01:39:45.334194+0000","last_active":"2023-10-06T01:39:49.350190+0000","last_peered":"2023-10-06T01:39:49.350190+0000","last_clean":"2023-10-06T01:39:49.350190+0000","last_became_active":"2023-10-06T01:39:45.333745+0000","last_became_peered":"2023-10-06T01:39:45.333745+0000","last_unstale":"2023-10-06T01:39:49.350190+0000","last_undegraded":"2023-10-06T01:39:49.350190+0000","last_fullsized":"2023-10-06T01:39:49.350190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.335131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360851+0000","last_change":"2023-10-06T01:39:49.359145+0000","last_active":"2023-10-06T01:39:49.360851+0000","last_peered":"2023-10-06T01:39:49.360851+0000","last_clean":"2023-10-06T01:39:49.360851+0000","last_became_active":"2023-10-06T01:39:49.358785+0000","last_became_peered":"2023-10-06T01:39:49.358785+0000","last_unstale":"2023-10-06T01:39:49.360851+0000","last_undegraded":"2023-10-06T01:39:49.360851+0000","last_fullsized":"2023-10-06T01:39:49.360851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:22.255025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340582+0000","last_change":"2023-10-06T01:39:44.325257+0000","last_active":"2023-10-06T01:39:49.340582+0000","last_peered":"2023-10-06T01:39:49.340582+0000","last_clean":"2023-10-06T01:39:49.340582+0000","last_became_active":"2023-10-06T01:39:44.324956+0000","last_became_peered":"2023-10-06T01:39:44.324956+0000","last_unstale":"2023-10-06T01:39:49.340582+0000","last_undegraded":"2023-10-06T01:39:49.340582+0000","last_fullsized":"2023-10-06T01:39:49.340582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338359+0000","last_change":"2023-10-06T01:39:45.331219+0000","last_active":"2023-10-06T01:39:49.338359+0000","last_peered":"2023-10-06T01:39:49.338359+0000","last_clean":"2023-10-06T01:39:49.338359+0000","last_became_active":"2023-10-06T01:39:45.330964+0000","last_became_peered":"2023-10-06T01:39:45.330964+0000","last_unstale":"2023-10-06T01:39:49.338359+0000","last_undegraded":"2023-10-06T01:39:49.338359+0000","last_fullsized":"2023-10-06T01:39:49.338359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359624+0000","last_change":"2023-10-06T01:39:49.356248+0000","last_active":"2023-10-06T01:39:49.359624+0000","last_peered":"2023-10-06T01:39:49.359624+0000","last_clean":"2023-10-06T01:39:49.359624+0000","last_became_active":"2023-10-06T01:39:49.355851+0000","last_became_peered":"2023-10-06T01:39:49.355851+0000","last_unstale":"2023-10-06T01:39:49.359624+0000","last_undegraded":"2023-10-06T01:39:49.359624+0000","last_fullsized":"2023-10-06T01:39:49.359624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341956+0000","last_change":"2023-10-06T01:39:44.327847+0000","last_active":"2023-10-06T01:39:49.341956+0000","last_peered":"2023-10-06T01:39:49.341956+0000","last_clean":"2023-10-06T01:39:49.341956+0000","last_became_active":"2023-10-06T01:39:44.327533+0000","last_became_peered":"2023-10-06T01:39:44.327533+0000","last_unstale":"2023-10-06T01:39:49.341956+0000","last_undegraded":"2023-10-06T01:39:49.341956+0000","last_fullsized":"2023-10-06T01:39:49.341956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351266+0000","last_change":"2023-10-06T01:39:45.326239+0000","last_active":"2023-10-06T01:39:49.351266+0000","last_peered":"2023-10-06T01:39:49.351266+0000","last_clean":"2023-10-06T01:39:49.351266+0000","last_became_active":"2023-10-06T01:39:45.325981+0000","last_became_peered":"2023-10-06T01:39:45.325981+0000","last_unstale":"2023-10-06T01:39:49.351266+0000","last_undegraded":"2023-10-06T01:39:49.351266+0000","last_fullsized":"2023-10-06T01:39:49.351266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361578+0000","last_change":"2023-10-06T01:39:49.358502+0000","last_active":"2023-10-06T01:39:49.361578+0000","last_peered":"2023-10-06T01:39:49.361578+0000","last_clean":"2023-10-06T01:39:49.361578+0000","last_became_active":"2023-10-06T01:39:49.358114+0000","last_became_peered":"2023-10-06T01:39:49.358114+0000","last_unstale":"2023-10-06T01:39:49.361578+0000","last_undegraded":"2023-10-06T01:39:49.361578+0000","last_fullsized":"2023-10-06T01:39:49.361578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338202+0000","last_change":"2023-10-06T01:39:44.325114+0000","last_active":"2023-10-06T01:39:49.338202+0000","last_peered":"2023-10-06T01:39:49.338202+0000","last_clean":"2023-10-06T01:39:49.338202+0000","last_became_active":"2023-10-06T01:39:44.324802+0000","last_became_peered":"2023-10-06T01:39:44.324802+0000","last_unstale":"2023-10-06T01:39:49.338202+0000","last_undegraded":"2023-10-06T01:39:49.338202+0000","last_fullsized":"2023-10-06T01:39:49.338202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:04:25.402016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339917+0000","last_change":"2023-10-06T01:39:45.337997+0000","last_active":"2023-10-06T01:39:49.339917+0000","last_peered":"2023-10-06T01:39:49.339917+0000","last_clean":"2023-10-06T01:39:49.339917+0000","last_became_active":"2023-10-06T01:39:45.337770+0000","last_became_peered":"2023-10-06T01:39:45.337770+0000","last_unstale":"2023-10-06T01:39:49.339917+0000","last_undegraded":"2023-10-06T01:39:49.339917+0000","last_fullsized":"2023-10-06T01:39:49.339917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359080+0000","last_change":"2023-10-06T01:39:49.355683+0000","last_active":"2023-10-06T01:39:49.359080+0000","last_peered":"2023-10-06T01:39:49.359080+0000","last_clean":"2023-10-06T01:39:49.359080+0000","last_became_active":"2023-10-06T01:39:49.355122+0000","last_became_peered":"2023-10-06T01:39:49.355122+0000","last_unstale":"2023-10-06T01:39:49.359080+0000","last_undegraded":"2023-10-06T01:39:49.359080+0000","last_fullsized":"2023-10-06T01:39:49.359080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351467+0000","last_change":"2023-10-06T01:39:44.326264+0000","last_active":"2023-10-06T01:39:49.351467+0000","last_peered":"2023-10-06T01:39:49.351467+0000","last_clean":"2023-10-06T01:39:49.351467+0000","last_became_active":"2023-10-06T01:39:44.325998+0000","last_became_peered":"2023-10-06T01:39:44.325998+0000","last_unstale":"2023-10-06T01:39:49.351467+0000","last_undegraded":"2023-10-06T01:39:49.351467+0000","last_fullsized":"2023-10-06T01:39:49.351467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338117+0000","last_change":"2023-10-06T01:39:45.332320+0000","last_active":"2023-10-06T01:39:49.338117+0000","last_peered":"2023-10-06T01:39:49.338117+0000","last_clean":"2023-10-06T01:39:49.338117+0000","last_became_active":"2023-10-06T01:39:45.332049+0000","last_became_peered":"2023-10-06T01:39:45.332049+0000","last_unstale":"2023-10-06T01:39:49.338117+0000","last_undegraded":"2023-10-06T01:39:49.338117+0000","last_fullsized":"2023-10-06T01:39:49.338117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363738+0000","last_change":"2023-10-06T01:39:49.360295+0000","last_active":"2023-10-06T01:39:49.363738+0000","last_peered":"2023-10-06T01:39:49.363738+0000","last_clean":"2023-10-06T01:39:49.363738+0000","last_became_active":"2023-10-06T01:39:49.359873+0000","last_became_peered":"2023-10-06T01:39:49.359873+0000","last_unstale":"2023-10-06T01:39:49.363738+0000","last_undegraded":"2023-10-06T01:39:49.363738+0000","last_fullsized":"2023-10-06T01:39:49.363738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356499+0000","last_change":"2023-10-06T01:39:44.328151+0000","last_active":"2023-10-06T01:39:49.356499+0000","last_peered":"2023-10-06T01:39:49.356499+0000","last_clean":"2023-10-06T01:39:49.356499+0000","last_became_active":"2023-10-06T01:39:44.327876+0000","last_became_peered":"2023-10-06T01:39:44.327876+0000","last_unstale":"2023-10-06T01:39:49.356499+0000","last_undegraded":"2023-10-06T01:39:49.356499+0000","last_fullsized":"2023-10-06T01:39:49.356499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337479+0000","last_change":"2023-10-06T01:39:45.330616+0000","last_active":"2023-10-06T01:39:49.337479+0000","last_peered":"2023-10-06T01:39:49.337479+0000","last_clean":"2023-10-06T01:39:49.337479+0000","last_became_active":"2023-10-06T01:39:45.330340+0000","last_became_peered":"2023-10-06T01:39:45.330340+0000","last_unstale":"2023-10-06T01:39:49.337479+0000","last_undegraded":"2023-10-06T01:39:49.337479+0000","last_fullsized":"2023-10-06T01:39:49.337479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368948+0000","last_change":"2023-10-06T01:39:49.365881+0000","last_active":"2023-10-06T01:39:49.368948+0000","last_peered":"2023-10-06T01:39:49.368948+0000","last_clean":"2023-10-06T01:39:49.368948+0000","last_became_active":"2023-10-06T01:39:49.365499+0000","last_became_peered":"2023-10-06T01:39:49.365499+0000","last_unstale":"2023-10-06T01:39:49.368948+0000","last_undegraded":"2023-10-06T01:39:49.368948+0000","last_fullsized":"2023-10-06T01:39:49.368948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:06.882659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351163+0000","last_change":"2023-10-06T01:39:44.326937+0000","last_active":"2023-10-06T01:39:49.351163+0000","last_peered":"2023-10-06T01:39:49.351163+0000","last_clean":"2023-10-06T01:39:49.351163+0000","last_became_active":"2023-10-06T01:39:44.326701+0000","last_became_peered":"2023-10-06T01:39:44.326701+0000","last_unstale":"2023-10-06T01:39:49.351163+0000","last_undegraded":"2023-10-06T01:39:49.351163+0000","last_fullsized":"2023-10-06T01:39:49.351163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:23.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342612+0000","last_change":"2023-10-06T01:39:45.331176+0000","last_active":"2023-10-06T01:39:49.342612+0000","last_peered":"2023-10-06T01:39:49.342612+0000","last_clean":"2023-10-06T01:39:49.342612+0000","last_became_active":"2023-10-06T01:39:45.330927+0000","last_became_peered":"2023-10-06T01:39:45.330927+0000","last_unstale":"2023-10-06T01:39:49.342612+0000","last_undegraded":"2023-10-06T01:39:49.342612+0000","last_fullsized":"2023-10-06T01:39:49.342612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357166+0000","last_change":"2023-10-06T01:39:49.355332+0000","last_active":"2023-10-06T01:39:49.357166+0000","last_peered":"2023-10-06T01:39:49.357166+0000","last_clean":"2023-10-06T01:39:49.357166+0000","last_became_active":"2023-10-06T01:39:49.354961+0000","last_became_peered":"2023-10-06T01:39:49.354961+0000","last_unstale":"2023-10-06T01:39:49.357166+0000","last_undegraded":"2023-10-06T01:39:49.357166+0000","last_fullsized":"2023-10-06T01:39:49.357166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:56.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T01:39:55.162081+0000","last_change":"2023-10-06T01:39:44.325901+0000","last_active":"2023-10-06T01:39:55.162081+0000","last_peered":"2023-10-06T01:39:55.162081+0000","last_clean":"2023-10-06T01:39:55.162081+0000","last_became_active":"2023-10-06T01:39:44.325631+0000","last_became_peered":"2023-10-06T01:39:44.325631+0000","last_unstale":"2023-10-06T01:39:55.162081+0000","last_undegraded":"2023-10-06T01:39:55.162081+0000","last_fullsized":"2023-10-06T01:39:55.162081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340321+0000","last_change":"2023-10-06T01:39:45.328644+0000","last_active":"2023-10-06T01:39:49.340321+0000","last_peered":"2023-10-06T01:39:49.340321+0000","last_clean":"2023-10-06T01:39:49.340321+0000","last_became_active":"2023-10-06T01:39:45.328367+0000","last_became_peered":"2023-10-06T01:39:45.328367+0000","last_unstale":"2023-10-06T01:39:49.340321+0000","last_undegraded":"2023-10-06T01:39:49.340321+0000","last_fullsized":"2023-10-06T01:39:49.340321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367347+0000","last_change":"2023-10-06T01:39:49.364574+0000","last_active":"2023-10-06T01:39:49.367347+0000","last_peered":"2023-10-06T01:39:49.367347+0000","last_clean":"2023-10-06T01:39:49.367347+0000","last_became_active":"2023-10-06T01:39:49.360749+0000","last_became_peered":"2023-10-06T01:39:49.360749+0000","last_unstale":"2023-10-06T01:39:49.367347+0000","last_undegraded":"2023-10-06T01:39:49.367347+0000","last_fullsized":"2023-10-06T01:39:49.367347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350570+0000","last_change":"2023-10-06T01:39:44.332362+0000","last_active":"2023-10-06T01:39:49.350570+0000","last_peered":"2023-10-06T01:39:49.350570+0000","last_clean":"2023-10-06T01:39:49.350570+0000","last_became_active":"2023-10-06T01:39:44.332082+0000","last_became_peered":"2023-10-06T01:39:44.332082+0000","last_unstale":"2023-10-06T01:39:49.350570+0000","last_undegraded":"2023-10-06T01:39:49.350570+0000","last_fullsized":"2023-10-06T01:39:49.350570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338444+0000","last_change":"2023-10-06T01:39:45.333142+0000","last_active":"2023-10-06T01:39:49.338444+0000","last_peered":"2023-10-06T01:39:49.338444+0000","last_clean":"2023-10-06T01:39:49.338444+0000","last_became_active":"2023-10-06T01:39:45.332880+0000","last_became_peered":"2023-10-06T01:39:45.332880+0000","last_unstale":"2023-10-06T01:39:49.338444+0000","last_undegraded":"2023-10-06T01:39:49.338444+0000","last_fullsized":"2023-10-06T01:39:49.338444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359341+0000","last_change":"2023-10-06T01:39:49.357412+0000","last_active":"2023-10-06T01:39:49.359341+0000","last_peered":"2023-10-06T01:39:49.359341+0000","last_clean":"2023-10-06T01:39:49.359341+0000","last_became_active":"2023-10-06T01:39:49.357031+0000","last_became_peered":"2023-10-06T01:39:49.357031+0000","last_unstale":"2023-10-06T01:39:49.359341+0000","last_undegraded":"2023-10-06T01:39:49.359341+0000","last_fullsized":"2023-10-06T01:39:49.359341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:34:21.212995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.348119+0000","last_change":"2023-10-06T01:39:44.325262+0000","last_active":"2023-10-06T01:39:49.348119+0000","last_peered":"2023-10-06T01:39:49.348119+0000","last_clean":"2023-10-06T01:39:49.348119+0000","last_became_active":"2023-10-06T01:39:44.325017+0000","last_became_peered":"2023-10-06T01:39:44.325017+0000","last_unstale":"2023-10-06T01:39:49.348119+0000","last_undegraded":"2023-10-06T01:39:49.348119+0000","last_fullsized":"2023-10-06T01:39:49.348119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350108+0000","last_change":"2023-10-06T01:39:45.334022+0000","last_active":"2023-10-06T01:39:49.350108+0000","last_peered":"2023-10-06T01:39:49.350108+0000","last_clean":"2023-10-06T01:39:49.350108+0000","last_became_active":"2023-10-06T01:39:45.333462+0000","last_became_peered":"2023-10-06T01:39:45.333462+0000","last_unstale":"2023-10-06T01:39:49.350108+0000","last_undegraded":"2023-10-06T01:39:49.350108+0000","last_fullsized":"2023-10-06T01:39:49.350108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.358648+0000","last_change":"2023-10-06T01:39:49.357484+0000","last_active":"2023-10-06T01:39:49.358648+0000","last_peered":"2023-10-06T01:39:49.358648+0000","last_clean":"2023-10-06T01:39:49.358648+0000","last_became_active":"2023-10-06T01:39:49.356839+0000","last_became_peered":"2023-10-06T01:39:49.356839+0000","last_unstale":"2023-10-06T01:39:49.358648+0000","last_undegraded":"2023-10-06T01:39:49.358648+0000","last_fullsized":"2023-10-06T01:39:49.358648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:14.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350742+0000","last_change":"2023-10-06T01:39:44.333226+0000","last_active":"2023-10-06T01:39:49.350742+0000","last_peered":"2023-10-06T01:39:49.350742+0000","last_clean":"2023-10-06T01:39:49.350742+0000","last_became_active":"2023-10-06T01:39:44.332920+0000","last_became_peered":"2023-10-06T01:39:44.332920+0000","last_unstale":"2023-10-06T01:39:49.350742+0000","last_undegraded":"2023-10-06T01:39:49.350742+0000","last_fullsized":"2023-10-06T01:39:49.350742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:09.344914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351529+0000","last_change":"2023-10-06T01:39:45.333240+0000","last_active":"2023-10-06T01:39:49.351529+0000","last_peered":"2023-10-06T01:39:49.351529+0000","last_clean":"2023-10-06T01:39:49.351529+0000","last_became_active":"2023-10-06T01:39:45.332985+0000","last_became_peered":"2023-10-06T01:39:45.332985+0000","last_unstale":"2023-10-06T01:39:49.351529+0000","last_undegraded":"2023-10-06T01:39:49.351529+0000","last_fullsized":"2023-10-06T01:39:49.351529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:10.908216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.364965+0000","last_change":"2023-10-06T01:39:49.360274+0000","last_active":"2023-10-06T01:39:49.364965+0000","last_peered":"2023-10-06T01:39:49.364965+0000","last_clean":"2023-10-06T01:39:49.364965+0000","last_became_active":"2023-10-06T01:39:49.359287+0000","last_became_peered":"2023-10-06T01:39:49.359287+0000","last_unstale":"2023-10-06T01:39:49.364965+0000","last_undegraded":"2023-10-06T01:39:49.364965+0000","last_fullsized":"2023-10-06T01:39:49.364965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:39.297464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343143+0000","last_change":"2023-10-06T01:39:44.322253+0000","last_active":"2023-10-06T01:39:49.343143+0000","last_peered":"2023-10-06T01:39:49.343143+0000","last_clean":"2023-10-06T01:39:49.343143+0000","last_became_active":"2023-10-06T01:39:44.321953+0000","last_became_peered":"2023-10-06T01:39:44.321953+0000","last_unstale":"2023-10-06T01:39:49.343143+0000","last_undegraded":"2023-10-06T01:39:49.343143+0000","last_fullsized":"2023-10-06T01:39:49.343143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338146+0000","last_change":"2023-10-06T01:39:45.343656+0000","last_active":"2023-10-06T01:39:49.338146+0000","last_peered":"2023-10-06T01:39:49.338146+0000","last_clean":"2023-10-06T01:39:49.338146+0000","last_became_active":"2023-10-06T01:39:45.342592+0000","last_became_peered":"2023-10-06T01:39:45.342592+0000","last_unstale":"2023-10-06T01:39:49.338146+0000","last_undegraded":"2023-10-06T01:39:49.338146+0000","last_fullsized":"2023-10-06T01:39:49.338146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365064+0000","last_change":"2023-10-06T01:39:49.363708+0000","last_active":"2023-10-06T01:39:49.365064+0000","last_peered":"2023-10-06T01:39:49.365064+0000","last_clean":"2023-10-06T01:39:49.365064+0000","last_became_active":"2023-10-06T01:39:49.363055+0000","last_became_peered":"2023-10-06T01:39:49.363055+0000","last_unstale":"2023-10-06T01:39:49.365064+0000","last_undegraded":"2023-10-06T01:39:49.365064+0000","last_fullsized":"2023-10-06T01:39:49.365064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352442+0000","last_change":"2023-10-06T01:39:44.333358+0000","last_active":"2023-10-06T01:39:49.352442+0000","last_peered":"2023-10-06T01:39:49.352442+0000","last_clean":"2023-10-06T01:39:49.352442+0000","last_became_active":"2023-10-06T01:39:44.332979+0000","last_became_peered":"2023-10-06T01:39:44.332979+0000","last_unstale":"2023-10-06T01:39:49.352442+0000","last_undegraded":"2023-10-06T01:39:49.352442+0000","last_fullsized":"2023-10-06T01:39:49.352442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349928+0000","last_change":"2023-10-06T01:39:45.330806+0000","last_active":"2023-10-06T01:39:49.349928+0000","last_peered":"2023-10-06T01:39:49.349928+0000","last_clean":"2023-10-06T01:39:49.349928+0000","last_became_active":"2023-10-06T01:39:45.330280+0000","last_became_peered":"2023-10-06T01:39:45.330280+0000","last_unstale":"2023-10-06T01:39:49.349928+0000","last_undegraded":"2023-10-06T01:39:49.349928+0000","last_fullsized":"2023-10-06T01:39:49.349928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.355843+0000","last_change":"2023-10-06T01:39:49.352785+0000","last_active":"2023-10-06T01:39:49.355843+0000","last_peered":"2023-10-06T01:39:49.355843+0000","last_clean":"2023-10-06T01:39:49.355843+0000","last_became_active":"2023-10-06T01:39:49.352396+0000","last_became_peered":"2023-10-06T01:39:49.352396+0000","last_unstale":"2023-10-06T01:39:49.355843+0000","last_undegraded":"2023-10-06T01:39:49.355843+0000","last_fullsized":"2023-10-06T01:39:49.355843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:45.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.352263+0000","last_change":"2023-10-06T01:39:44.332277+0000","last_active":"2023-10-06T01:39:49.352263+0000","last_peered":"2023-10-06T01:39:49.352263+0000","last_clean":"2023-10-06T01:39:49.352263+0000","last_became_active":"2023-10-06T01:39:44.331987+0000","last_became_peered":"2023-10-06T01:39:44.331987+0000","last_unstale":"2023-10-06T01:39:49.352263+0000","last_undegraded":"2023-10-06T01:39:49.352263+0000","last_fullsized":"2023-10-06T01:39:49.352263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:36.012185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342030+0000","last_change":"2023-10-06T01:39:45.326893+0000","last_active":"2023-10-06T01:39:49.342030+0000","last_peered":"2023-10-06T01:39:49.342030+0000","last_clean":"2023-10-06T01:39:49.342030+0000","last_became_active":"2023-10-06T01:39:45.326610+0000","last_became_peered":"2023-10-06T01:39:45.326610+0000","last_unstale":"2023-10-06T01:39:49.342030+0000","last_undegraded":"2023-10-06T01:39:49.342030+0000","last_fullsized":"2023-10-06T01:39:49.342030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.723412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.360816+0000","last_change":"2023-10-06T01:39:49.359772+0000","last_active":"2023-10-06T01:39:49.360816+0000","last_peered":"2023-10-06T01:39:49.360816+0000","last_clean":"2023-10-06T01:39:49.360816+0000","last_became_active":"2023-10-06T01:39:49.359414+0000","last_became_peered":"2023-10-06T01:39:49.359414+0000","last_unstale":"2023-10-06T01:39:49.360816+0000","last_undegraded":"2023-10-06T01:39:49.360816+0000","last_fullsized":"2023-10-06T01:39:49.360816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:27.073697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351061+0000","last_change":"2023-10-06T01:39:44.330033+0000","last_active":"2023-10-06T01:39:49.351061+0000","last_peered":"2023-10-06T01:39:49.351061+0000","last_clean":"2023-10-06T01:39:49.351061+0000","last_became_active":"2023-10-06T01:39:44.329732+0000","last_became_peered":"2023-10-06T01:39:44.329732+0000","last_unstale":"2023-10-06T01:39:49.351061+0000","last_undegraded":"2023-10-06T01:39:49.351061+0000","last_fullsized":"2023-10-06T01:39:49.351061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339868+0000","last_change":"2023-10-06T01:39:45.334185+0000","last_active":"2023-10-06T01:39:49.339868+0000","last_peered":"2023-10-06T01:39:49.339868+0000","last_clean":"2023-10-06T01:39:49.339868+0000","last_became_active":"2023-10-06T01:39:45.333053+0000","last_became_peered":"2023-10-06T01:39:45.333053+0000","last_unstale":"2023-10-06T01:39:49.339868+0000","last_undegraded":"2023-10-06T01:39:49.339868+0000","last_fullsized":"2023-10-06T01:39:49.339868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:58.429853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356192+0000","last_change":"2023-10-06T01:39:49.353291+0000","last_active":"2023-10-06T01:39:49.356192+0000","last_peered":"2023-10-06T01:39:49.356192+0000","last_clean":"2023-10-06T01:39:49.356192+0000","last_became_active":"2023-10-06T01:39:49.352912+0000","last_became_peered":"2023-10-06T01:39:49.352912+0000","last_unstale":"2023-10-06T01:39:49.356192+0000","last_undegraded":"2023-10-06T01:39:49.356192+0000","last_fullsized":"2023-10-06T01:39:49.356192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.229209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.325148+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.324687+0000","last_became_peered":"2023-10-06T01:39:44.324687+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347753+0000","last_change":"2023-10-06T01:39:45.329116+0000","last_active":"2023-10-06T01:39:49.347753+0000","last_peered":"2023-10-06T01:39:49.347753+0000","last_clean":"2023-10-06T01:39:49.347753+0000","last_became_active":"2023-10-06T01:39:45.328839+0000","last_became_peered":"2023-10-06T01:39:45.328839+0000","last_unstale":"2023-10-06T01:39:49.347753+0000","last_undegraded":"2023-10-06T01:39:49.347753+0000","last_fullsized":"2023-10-06T01:39:49.347753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:50.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356181+0000","last_change":"2023-10-06T01:39:49.353796+0000","last_active":"2023-10-06T01:39:49.356181+0000","last_peered":"2023-10-06T01:39:49.356181+0000","last_clean":"2023-10-06T01:39:49.356181+0000","last_became_active":"2023-10-06T01:39:49.353138+0000","last_became_peered":"2023-10-06T01:39:49.353138+0000","last_unstale":"2023-10-06T01:39:49.356181+0000","last_undegraded":"2023-10-06T01:39:49.356181+0000","last_fullsized":"2023-10-06T01:39:49.356181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337716+0000","last_change":"2023-10-06T01:39:44.329032+0000","last_active":"2023-10-06T01:39:49.337716+0000","last_peered":"2023-10-06T01:39:49.337716+0000","last_clean":"2023-10-06T01:39:49.337716+0000","last_became_active":"2023-10-06T01:39:44.328775+0000","last_became_peered":"2023-10-06T01:39:44.328775+0000","last_unstale":"2023-10-06T01:39:49.337716+0000","last_undegraded":"2023-10-06T01:39:49.337716+0000","last_fullsized":"2023-10-06T01:39:49.337716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:34.126496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331469+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.331152+0000","last_became_peered":"2023-10-06T01:39:45.331152+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:08.669768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356783+0000","last_change":"2023-10-06T01:39:49.351619+0000","last_active":"2023-10-06T01:39:49.356783+0000","last_peered":"2023-10-06T01:39:49.356783+0000","last_clean":"2023-10-06T01:39:49.356783+0000","last_became_active":"2023-10-06T01:39:49.351271+0000","last_became_peered":"2023-10-06T01:39:49.351271+0000","last_unstale":"2023-10-06T01:39:49.356783+0000","last_undegraded":"2023-10-06T01:39:49.356783+0000","last_fullsized":"2023-10-06T01:39:49.356783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:05.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339967+0000","last_change":"2023-10-06T01:39:44.321209+0000","last_active":"2023-10-06T01:39:49.339967+0000","last_peered":"2023-10-06T01:39:49.339967+0000","last_clean":"2023-10-06T01:39:49.339967+0000","last_became_active":"2023-10-06T01:39:44.320932+0000","last_became_peered":"2023-10-06T01:39:44.320932+0000","last_unstale":"2023-10-06T01:39:49.339967+0000","last_undegraded":"2023-10-06T01:39:49.339967+0000","last_fullsized":"2023-10-06T01:39:49.339967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339741+0000","last_change":"2023-10-06T01:39:45.339859+0000","last_active":"2023-10-06T01:39:49.339741+0000","last_peered":"2023-10-06T01:39:49.339741+0000","last_clean":"2023-10-06T01:39:49.339741+0000","last_became_active":"2023-10-06T01:39:45.339422+0000","last_became_peered":"2023-10-06T01:39:45.339422+0000","last_unstale":"2023-10-06T01:39:49.339741+0000","last_undegraded":"2023-10-06T01:39:49.339741+0000","last_fullsized":"2023-10-06T01:39:49.339741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:36.266399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367304+0000","last_change":"2023-10-06T01:39:49.361654+0000","last_active":"2023-10-06T01:39:49.367304+0000","last_peered":"2023-10-06T01:39:49.367304+0000","last_clean":"2023-10-06T01:39:49.367304+0000","last_became_active":"2023-10-06T01:39:49.361257+0000","last_became_peered":"2023-10-06T01:39:49.361257+0000","last_unstale":"2023-10-06T01:39:49.367304+0000","last_undegraded":"2023-10-06T01:39:49.367304+0000","last_fullsized":"2023-10-06T01:39:49.367304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:13:40.277782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341922+0000","last_change":"2023-10-06T01:39:44.326380+0000","last_active":"2023-10-06T01:39:49.341922+0000","last_peered":"2023-10-06T01:39:49.341922+0000","last_clean":"2023-10-06T01:39:49.341922+0000","last_became_active":"2023-10-06T01:39:44.326059+0000","last_became_peered":"2023-10-06T01:39:44.326059+0000","last_unstale":"2023-10-06T01:39:49.341922+0000","last_undegraded":"2023-10-06T01:39:49.341922+0000","last_fullsized":"2023-10-06T01:39:49.341922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342079+0000","last_change":"2023-10-06T01:39:45.329138+0000","last_active":"2023-10-06T01:39:49.342079+0000","last_peered":"2023-10-06T01:39:49.342079+0000","last_clean":"2023-10-06T01:39:49.342079+0000","last_became_active":"2023-10-06T01:39:45.328861+0000","last_became_peered":"2023-10-06T01:39:45.328861+0000","last_unstale":"2023-10-06T01:39:49.342079+0000","last_undegraded":"2023-10-06T01:39:49.342079+0000","last_fullsized":"2023-10-06T01:39:49.342079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.365004+0000","last_change":"2023-10-06T01:39:49.360752+0000","last_active":"2023-10-06T01:39:49.365004+0000","last_peered":"2023-10-06T01:39:49.365004+0000","last_clean":"2023-10-06T01:39:49.365004+0000","last_became_active":"2023-10-06T01:39:49.360376+0000","last_became_peered":"2023-10-06T01:39:49.360376+0000","last_unstale":"2023-10-06T01:39:49.365004+0000","last_undegraded":"2023-10-06T01:39:49.365004+0000","last_fullsized":"2023-10-06T01:39:49.365004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:28.092106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338022+0000","last_change":"2023-10-06T01:39:44.325482+0000","last_active":"2023-10-06T01:39:49.338022+0000","last_peered":"2023-10-06T01:39:49.338022+0000","last_clean":"2023-10-06T01:39:49.338022+0000","last_became_active":"2023-10-06T01:39:44.325160+0000","last_became_peered":"2023-10-06T01:39:44.325160+0000","last_unstale":"2023-10-06T01:39:49.338022+0000","last_undegraded":"2023-10-06T01:39:49.338022+0000","last_fullsized":"2023-10-06T01:39:49.338022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344360+0000","last_change":"2023-10-06T01:39:45.342331+0000","last_active":"2023-10-06T01:39:49.344360+0000","last_peered":"2023-10-06T01:39:49.344360+0000","last_clean":"2023-10-06T01:39:49.344360+0000","last_became_active":"2023-10-06T01:39:45.342022+0000","last_became_peered":"2023-10-06T01:39:45.342022+0000","last_unstale":"2023-10-06T01:39:49.344360+0000","last_undegraded":"2023-10-06T01:39:49.344360+0000","last_fullsized":"2023-10-06T01:39:49.344360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.201677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357018+0000","last_change":"2023-10-06T01:39:49.355423+0000","last_active":"2023-10-06T01:39:49.357018+0000","last_peered":"2023-10-06T01:39:49.357018+0000","last_clean":"2023-10-06T01:39:49.357018+0000","last_became_active":"2023-10-06T01:39:49.355075+0000","last_became_peered":"2023-10-06T01:39:49.355075+0000","last_unstale":"2023-10-06T01:39:49.357018+0000","last_undegraded":"2023-10-06T01:39:49.357018+0000","last_fullsized":"2023-10-06T01:39:49.357018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:02.539610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.338012+0000","last_change":"2023-10-06T01:39:44.328632+0000","last_active":"2023-10-06T01:39:49.338012+0000","last_peered":"2023-10-06T01:39:49.338012+0000","last_clean":"2023-10-06T01:39:49.338012+0000","last_became_active":"2023-10-06T01:39:44.326328+0000","last_became_peered":"2023-10-06T01:39:44.326328+0000","last_unstale":"2023-10-06T01:39:49.338012+0000","last_undegraded":"2023-10-06T01:39:49.338012+0000","last_fullsized":"2023-10-06T01:39:49.338012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339913+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:39:49.339913+0000","last_peered":"2023-10-06T01:39:49.339913+0000","last_clean":"2023-10-06T01:39:49.339913+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:39:49.339913+0000","last_undegraded":"2023-10-06T01:39:49.339913+0000","last_fullsized":"2023-10-06T01:39:49.339913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337712+0000","last_change":"2023-10-06T01:39:45.342039+0000","last_active":"2023-10-06T01:39:49.337712+0000","last_peered":"2023-10-06T01:39:49.337712+0000","last_clean":"2023-10-06T01:39:49.337712+0000","last_became_active":"2023-10-06T01:39:45.341746+0000","last_became_peered":"2023-10-06T01:39:45.341746+0000","last_unstale":"2023-10-06T01:39:49.337712+0000","last_undegraded":"2023-10-06T01:39:49.337712+0000","last_fullsized":"2023-10-06T01:39:49.337712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:51.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354087+0000","last_change":"2023-10-06T01:39:49.352264+0000","last_active":"2023-10-06T01:39:49.354087+0000","last_peered":"2023-10-06T01:39:49.354087+0000","last_clean":"2023-10-06T01:39:49.354087+0000","last_became_active":"2023-10-06T01:39:49.351974+0000","last_became_peered":"2023-10-06T01:39:49.351974+0000","last_unstale":"2023-10-06T01:39:49.354087+0000","last_undegraded":"2023-10-06T01:39:49.354087+0000","last_fullsized":"2023-10-06T01:39:49.354087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:35.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337764+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:39:49.337764+0000","last_peered":"2023-10-06T01:39:49.337764+0000","last_clean":"2023-10-06T01:39:49.337764+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:39:49.337764+0000","last_undegraded":"2023-10-06T01:39:49.337764+0000","last_fullsized":"2023-10-06T01:39:49.337764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.395905+0000","last_change":"2023-10-06T01:39:44.324833+0000","last_active":"2023-10-06T01:39:49.395905+0000","last_peered":"2023-10-06T01:39:49.395905+0000","last_clean":"2023-10-06T01:39:49.395905+0000","last_became_active":"2023-10-06T01:39:44.324554+0000","last_became_peered":"2023-10-06T01:39:44.324554+0000","last_unstale":"2023-10-06T01:39:49.395905+0000","last_undegraded":"2023-10-06T01:39:49.395905+0000","last_fullsized":"2023-10-06T01:39:49.395905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:53.581079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341933+0000","last_change":"2023-10-06T01:39:45.331099+0000","last_active":"2023-10-06T01:39:49.341933+0000","last_peered":"2023-10-06T01:39:49.341933+0000","last_clean":"2023-10-06T01:39:49.341933+0000","last_became_active":"2023-10-06T01:39:45.330844+0000","last_became_peered":"2023-10-06T01:39:45.330844+0000","last_unstale":"2023-10-06T01:39:49.341933+0000","last_undegraded":"2023-10-06T01:39:49.341933+0000","last_fullsized":"2023-10-06T01:39:49.341933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.367282+0000","last_change":"2023-10-06T01:39:49.360839+0000","last_active":"2023-10-06T01:39:49.367282+0000","last_peered":"2023-10-06T01:39:49.367282+0000","last_clean":"2023-10-06T01:39:49.367282+0000","last_became_active":"2023-10-06T01:39:49.360447+0000","last_became_peered":"2023-10-06T01:39:49.360447+0000","last_unstale":"2023-10-06T01:39:49.367282+0000","last_undegraded":"2023-10-06T01:39:49.367282+0000","last_fullsized":"2023-10-06T01:39:49.367282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342223+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:39:49.342223+0000","last_peered":"2023-10-06T01:39:49.342223+0000","last_clean":"2023-10-06T01:39:49.342223+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:39:49.342223+0000","last_undegraded":"2023-10-06T01:39:49.342223+0000","last_fullsized":"2023-10-06T01:39:49.342223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350783+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:39:49.350783+0000","last_peered":"2023-10-06T01:39:49.350783+0000","last_clean":"2023-10-06T01:39:49.350783+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:39:49.350783+0000","last_undegraded":"2023-10-06T01:39:49.350783+0000","last_fullsized":"2023-10-06T01:39:49.350783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346294+0000","last_change":"2023-10-06T01:39:44.332942+0000","last_active":"2023-10-06T01:39:49.346294+0000","last_peered":"2023-10-06T01:39:49.346294+0000","last_clean":"2023-10-06T01:39:49.346294+0000","last_became_active":"2023-10-06T01:39:44.332767+0000","last_became_peered":"2023-10-06T01:39:44.332767+0000","last_unstale":"2023-10-06T01:39:49.346294+0000","last_undegraded":"2023-10-06T01:39:49.346294+0000","last_fullsized":"2023-10-06T01:39:49.346294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351451+0000","last_change":"2023-10-06T01:39:45.331974+0000","last_active":"2023-10-06T01:39:49.351451+0000","last_peered":"2023-10-06T01:39:49.351451+0000","last_clean":"2023-10-06T01:39:49.351451+0000","last_became_active":"2023-10-06T01:39:45.331706+0000","last_became_peered":"2023-10-06T01:39:45.331706+0000","last_unstale":"2023-10-06T01:39:49.351451+0000","last_undegraded":"2023-10-06T01:39:49.351451+0000","last_fullsized":"2023-10-06T01:39:49.351451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.357328+0000","last_change":"2023-10-06T01:39:49.356514+0000","last_active":"2023-10-06T01:39:49.357328+0000","last_peered":"2023-10-06T01:39:49.357328+0000","last_clean":"2023-10-06T01:39:49.357328+0000","last_became_active":"2023-10-06T01:39:49.356017+0000","last_became_peered":"2023-10-06T01:39:49.356017+0000","last_unstale":"2023-10-06T01:39:49.357328+0000","last_undegraded":"2023-10-06T01:39:49.357328+0000","last_fullsized":"2023-10-06T01:39:49.357328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:28.616739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350977+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:39:49.350977+0000","last_peered":"2023-10-06T01:39:49.350977+0000","last_clean":"2023-10-06T01:39:49.350977+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:39:49.350977+0000","last_undegraded":"2023-10-06T01:39:49.350977+0000","last_fullsized":"2023-10-06T01:39:49.350977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.345374+0000","last_change":"2023-10-06T01:39:44.324538+0000","last_active":"2023-10-06T01:39:49.345374+0000","last_peered":"2023-10-06T01:39:49.345374+0000","last_clean":"2023-10-06T01:39:49.345374+0000","last_became_active":"2023-10-06T01:39:44.324239+0000","last_became_peered":"2023-10-06T01:39:44.324239+0000","last_unstale":"2023-10-06T01:39:49.345374+0000","last_undegraded":"2023-10-06T01:39:49.345374+0000","last_fullsized":"2023-10-06T01:39:49.345374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.344282+0000","last_change":"2023-10-06T01:39:45.339432+0000","last_active":"2023-10-06T01:39:49.344282+0000","last_peered":"2023-10-06T01:39:49.344282+0000","last_clean":"2023-10-06T01:39:49.344282+0000","last_became_active":"2023-10-06T01:39:45.339191+0000","last_became_peered":"2023-10-06T01:39:45.339191+0000","last_unstale":"2023-10-06T01:39:49.344282+0000","last_undegraded":"2023-10-06T01:39:49.344282+0000","last_fullsized":"2023-10-06T01:39:49.344282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363746+0000","last_change":"2023-10-06T01:39:49.361536+0000","last_active":"2023-10-06T01:39:49.363746+0000","last_peered":"2023-10-06T01:39:49.363746+0000","last_clean":"2023-10-06T01:39:49.363746+0000","last_became_active":"2023-10-06T01:39:49.361162+0000","last_became_peered":"2023-10-06T01:39:49.361162+0000","last_unstale":"2023-10-06T01:39:49.363746+0000","last_undegraded":"2023-10-06T01:39:49.363746+0000","last_fullsized":"2023-10-06T01:39:49.363746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350576+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:39:49.350576+0000","last_peered":"2023-10-06T01:39:49.350576+0000","last_clean":"2023-10-06T01:39:49.350576+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:39:49.350576+0000","last_undegraded":"2023-10-06T01:39:49.350576+0000","last_fullsized":"2023-10-06T01:39:49.350576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337530+0000","last_change":"2023-10-06T01:39:44.328264+0000","last_active":"2023-10-06T01:39:49.337530+0000","last_peered":"2023-10-06T01:39:49.337530+0000","last_clean":"2023-10-06T01:39:49.337530+0000","last_became_active":"2023-10-06T01:39:44.327969+0000","last_became_peered":"2023-10-06T01:39:44.327969+0000","last_unstale":"2023-10-06T01:39:49.337530+0000","last_undegraded":"2023-10-06T01:39:49.337530+0000","last_fullsized":"2023-10-06T01:39:49.337530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:57.694734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340324+0000","last_change":"2023-10-06T01:39:45.333753+0000","last_active":"2023-10-06T01:39:49.340324+0000","last_peered":"2023-10-06T01:39:49.340324+0000","last_clean":"2023-10-06T01:39:49.340324+0000","last_became_active":"2023-10-06T01:39:45.333416+0000","last_became_peered":"2023-10-06T01:39:45.333416+0000","last_unstale":"2023-10-06T01:39:49.340324+0000","last_undegraded":"2023-10-06T01:39:49.340324+0000","last_fullsized":"2023-10-06T01:39:49.340324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.053563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.361707+0000","last_change":"2023-10-06T01:39:49.360037+0000","last_active":"2023-10-06T01:39:49.361707+0000","last_peered":"2023-10-06T01:39:49.361707+0000","last_clean":"2023-10-06T01:39:49.361707+0000","last_became_active":"2023-10-06T01:39:49.359697+0000","last_became_peered":"2023-10-06T01:39:49.359697+0000","last_unstale":"2023-10-06T01:39:49.361707+0000","last_undegraded":"2023-10-06T01:39:49.361707+0000","last_fullsized":"2023-10-06T01:39:49.361707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:25.915864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349757+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:39:49.349757+0000","last_peered":"2023-10-06T01:39:49.349757+0000","last_clean":"2023-10-06T01:39:49.349757+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:39:49.349757+0000","last_undegraded":"2023-10-06T01:39:49.349757+0000","last_fullsized":"2023-10-06T01:39:49.349757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340331+0000","last_change":"2023-10-06T01:39:44.329991+0000","last_active":"2023-10-06T01:39:49.340331+0000","last_peered":"2023-10-06T01:39:49.340331+0000","last_clean":"2023-10-06T01:39:49.340331+0000","last_became_active":"2023-10-06T01:39:44.329756+0000","last_became_peered":"2023-10-06T01:39:44.329756+0000","last_unstale":"2023-10-06T01:39:49.340331+0000","last_undegraded":"2023-10-06T01:39:49.340331+0000","last_fullsized":"2023-10-06T01:39:49.340331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343619+0000","last_change":"2023-10-06T01:39:45.336684+0000","last_active":"2023-10-06T01:39:49.343619+0000","last_peered":"2023-10-06T01:39:49.343619+0000","last_clean":"2023-10-06T01:39:49.343619+0000","last_became_active":"2023-10-06T01:39:45.336427+0000","last_became_peered":"2023-10-06T01:39:45.336427+0000","last_unstale":"2023-10-06T01:39:49.343619+0000","last_undegraded":"2023-10-06T01:39:49.343619+0000","last_fullsized":"2023-10-06T01:39:49.343619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:47:24.701422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.359866+0000","last_change":"2023-10-06T01:39:49.358138+0000","last_active":"2023-10-06T01:39:49.359866+0000","last_peered":"2023-10-06T01:39:49.359866+0000","last_clean":"2023-10-06T01:39:49.359866+0000","last_became_active":"2023-10-06T01:39:49.356144+0000","last_became_peered":"2023-10-06T01:39:49.356144+0000","last_unstale":"2023-10-06T01:39:49.359866+0000","last_undegraded":"2023-10-06T01:39:49.359866+0000","last_fullsized":"2023-10-06T01:39:49.359866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:02.622952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.341234+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:39:49.341234+0000","last_peered":"2023-10-06T01:39:49.341234+0000","last_clean":"2023-10-06T01:39:49.341234+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:39:49.341234+0000","last_undegraded":"2023-10-06T01:39:49.341234+0000","last_fullsized":"2023-10-06T01:39:49.341234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343734+0000","last_change":"2023-10-06T01:39:44.327413+0000","last_active":"2023-10-06T01:39:49.343734+0000","last_peered":"2023-10-06T01:39:49.343734+0000","last_clean":"2023-10-06T01:39:49.343734+0000","last_became_active":"2023-10-06T01:39:44.327177+0000","last_became_peered":"2023-10-06T01:39:44.327177+0000","last_unstale":"2023-10-06T01:39:49.343734+0000","last_undegraded":"2023-10-06T01:39:49.343734+0000","last_fullsized":"2023-10-06T01:39:49.343734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:17.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347454+0000","last_change":"2023-10-06T01:39:45.334858+0000","last_active":"2023-10-06T01:39:49.347454+0000","last_peered":"2023-10-06T01:39:49.347454+0000","last_clean":"2023-10-06T01:39:49.347454+0000","last_became_active":"2023-10-06T01:39:45.334586+0000","last_became_peered":"2023-10-06T01:39:45.334586+0000","last_unstale":"2023-10-06T01:39:49.347454+0000","last_undegraded":"2023-10-06T01:39:49.347454+0000","last_fullsized":"2023-10-06T01:39:49.347454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:10.350398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.366854+0000","last_change":"2023-10-06T01:39:49.365711+0000","last_active":"2023-10-06T01:39:49.366854+0000","last_peered":"2023-10-06T01:39:49.366854+0000","last_clean":"2023-10-06T01:39:49.366854+0000","last_became_active":"2023-10-06T01:39:49.365357+0000","last_became_peered":"2023-10-06T01:39:49.365357+0000","last_unstale":"2023-10-06T01:39:49.366854+0000","last_undegraded":"2023-10-06T01:39:49.366854+0000","last_fullsized":"2023-10-06T01:39:49.366854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.339829+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:39:49.339829+0000","last_peered":"2023-10-06T01:39:49.339829+0000","last_clean":"2023-10-06T01:39:49.339829+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:39:49.339829+0000","last_undegraded":"2023-10-06T01:39:49.339829+0000","last_fullsized":"2023-10-06T01:39:49.339829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.350343+0000","last_change":"2023-10-06T01:39:44.327795+0000","last_active":"2023-10-06T01:39:49.350343+0000","last_peered":"2023-10-06T01:39:49.350343+0000","last_clean":"2023-10-06T01:39:49.350343+0000","last_became_active":"2023-10-06T01:39:44.326791+0000","last_became_peered":"2023-10-06T01:39:44.326791+0000","last_unstale":"2023-10-06T01:39:49.350343+0000","last_undegraded":"2023-10-06T01:39:49.350343+0000","last_fullsized":"2023-10-06T01:39:49.350343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:36.085435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342538+0000","last_change":"2023-10-06T01:39:45.341418+0000","last_active":"2023-10-06T01:39:49.342538+0000","last_peered":"2023-10-06T01:39:49.342538+0000","last_clean":"2023-10-06T01:39:49.342538+0000","last_became_active":"2023-10-06T01:39:45.341053+0000","last_became_peered":"2023-10-06T01:39:45.341053+0000","last_unstale":"2023-10-06T01:39:49.342538+0000","last_undegraded":"2023-10-06T01:39:49.342538+0000","last_fullsized":"2023-10-06T01:39:49.342538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.470236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.356900+0000","last_change":"2023-10-06T01:39:49.355119+0000","last_active":"2023-10-06T01:39:49.356900+0000","last_peered":"2023-10-06T01:39:49.356900+0000","last_clean":"2023-10-06T01:39:49.356900+0000","last_became_active":"2023-10-06T01:39:49.354770+0000","last_became_peered":"2023-10-06T01:39:49.354770+0000","last_unstale":"2023-10-06T01:39:49.356900+0000","last_undegraded":"2023-10-06T01:39:49.356900+0000","last_fullsized":"2023-10-06T01:39:49.356900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:40.295194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.347370+0000","last_change":"2023-10-06T01:39:44.333929+0000","last_active":"2023-10-06T01:39:49.347370+0000","last_peered":"2023-10-06T01:39:49.347370+0000","last_clean":"2023-10-06T01:39:49.347370+0000","last_became_active":"2023-10-06T01:39:44.333659+0000","last_became_peered":"2023-10-06T01:39:44.333659+0000","last_unstale":"2023-10-06T01:39:49.347370+0000","last_undegraded":"2023-10-06T01:39:49.347370+0000","last_fullsized":"2023-10-06T01:39:49.347370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.346317+0000","last_change":"2023-10-06T01:39:45.331228+0000","last_active":"2023-10-06T01:39:49.346317+0000","last_peered":"2023-10-06T01:39:49.346317+0000","last_clean":"2023-10-06T01:39:49.346317+0000","last_became_active":"2023-10-06T01:39:45.330982+0000","last_became_peered":"2023-10-06T01:39:45.330982+0000","last_unstale":"2023-10-06T01:39:49.346317+0000","last_undegraded":"2023-10-06T01:39:49.346317+0000","last_fullsized":"2023-10-06T01:39:49.346317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.923135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.363774+0000","last_change":"2023-10-06T01:39:49.362572+0000","last_active":"2023-10-06T01:39:49.363774+0000","last_peered":"2023-10-06T01:39:49.363774+0000","last_clean":"2023-10-06T01:39:49.363774+0000","last_became_active":"2023-10-06T01:39:49.362200+0000","last_became_peered":"2023-10-06T01:39:49.362200+0000","last_unstale":"2023-10-06T01:39:49.363774+0000","last_undegraded":"2023-10-06T01:39:49.363774+0000","last_fullsized":"2023-10-06T01:39:49.363774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:25.170990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349796+0000","last_change":"2023-10-06T01:39:44.332451+0000","last_active":"2023-10-06T01:39:49.349796+0000","last_peered":"2023-10-06T01:39:49.349796+0000","last_clean":"2023-10-06T01:39:49.349796+0000","last_became_active":"2023-10-06T01:39:44.332143+0000","last_became_peered":"2023-10-06T01:39:44.332143+0000","last_unstale":"2023-10-06T01:39:49.349796+0000","last_undegraded":"2023-10-06T01:39:49.349796+0000","last_fullsized":"2023-10-06T01:39:49.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.257109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340132+0000","last_change":"2023-10-06T01:39:45.331260+0000","last_active":"2023-10-06T01:39:49.340132+0000","last_peered":"2023-10-06T01:39:49.340132+0000","last_clean":"2023-10-06T01:39:49.340132+0000","last_became_active":"2023-10-06T01:39:45.330980+0000","last_became_peered":"2023-10-06T01:39:45.330980+0000","last_unstale":"2023-10-06T01:39:49.340132+0000","last_undegraded":"2023-10-06T01:39:49.340132+0000","last_fullsized":"2023-10-06T01:39:49.340132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:41.594142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.368960+0000","last_change":"2023-10-06T01:39:49.361125+0000","last_active":"2023-10-06T01:39:49.368960+0000","last_peered":"2023-10-06T01:39:49.368960+0000","last_clean":"2023-10-06T01:39:49.368960+0000","last_became_active":"2023-10-06T01:39:49.354967+0000","last_became_peered":"2023-10-06T01:39:49.354967+0000","last_unstale":"2023-10-06T01:39:49.368960+0000","last_undegraded":"2023-10-06T01:39:49.368960+0000","last_fullsized":"2023-10-06T01:39:49.368960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:41.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.349958+0000","last_change":"2023-10-06T01:39:44.329353+0000","last_active":"2023-10-06T01:39:49.349958+0000","last_peered":"2023-10-06T01:39:49.349958+0000","last_clean":"2023-10-06T01:39:49.349958+0000","last_became_active":"2023-10-06T01:39:44.329098+0000","last_became_peered":"2023-10-06T01:39:44.329098+0000","last_unstale":"2023-10-06T01:39:49.349958+0000","last_undegraded":"2023-10-06T01:39:49.349958+0000","last_fullsized":"2023-10-06T01:39:49.349958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340491+0000","last_change":"2023-10-06T01:39:45.330780+0000","last_active":"2023-10-06T01:39:49.340491+0000","last_peered":"2023-10-06T01:39:49.340491+0000","last_clean":"2023-10-06T01:39:49.340491+0000","last_became_active":"2023-10-06T01:39:45.330484+0000","last_became_peered":"2023-10-06T01:39:45.330484+0000","last_unstale":"2023-10-06T01:39:49.340491+0000","last_undegraded":"2023-10-06T01:39:49.340491+0000","last_fullsized":"2023-10-06T01:39:49.340491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:28.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.354659+0000","last_change":"2023-10-06T01:39:49.351706+0000","last_active":"2023-10-06T01:39:49.354659+0000","last_peered":"2023-10-06T01:39:49.354659+0000","last_clean":"2023-10-06T01:39:49.354659+0000","last_became_active":"2023-10-06T01:39:49.350630+0000","last_became_peered":"2023-10-06T01:39:49.350630+0000","last_unstale":"2023-10-06T01:39:49.354659+0000","last_undegraded":"2023-10-06T01:39:49.354659+0000","last_fullsized":"2023-10-06T01:39:49.354659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:57.336637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337644+0000","last_change":"2023-10-06T01:39:44.321387+0000","last_active":"2023-10-06T01:39:49.337644+0000","last_peered":"2023-10-06T01:39:49.337644+0000","last_clean":"2023-10-06T01:39:49.337644+0000","last_became_active":"2023-10-06T01:39:44.321101+0000","last_became_peered":"2023-10-06T01:39:44.321101+0000","last_unstale":"2023-10-06T01:39:49.337644+0000","last_undegraded":"2023-10-06T01:39:49.337644+0000","last_fullsized":"2023-10-06T01:39:49.337644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:25:12.552978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337678+0000","last_change":"2023-10-06T01:39:45.333689+0000","last_active":"2023-10-06T01:39:49.337678+0000","last_peered":"2023-10-06T01:39:49.337678+0000","last_clean":"2023-10-06T01:39:49.337678+0000","last_became_active":"2023-10-06T01:39:45.333444+0000","last_became_peered":"2023-10-06T01:39:45.333444+0000","last_unstale":"2023-10-06T01:39:49.337678+0000","last_undegraded":"2023-10-06T01:39:49.337678+0000","last_fullsized":"2023-10-06T01:39:49.337678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:32.004574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.362749+0000","last_change":"2023-10-06T01:39:49.360875+0000","last_active":"2023-10-06T01:39:49.362749+0000","last_peered":"2023-10-06T01:39:49.362749+0000","last_clean":"2023-10-06T01:39:49.362749+0000","last_became_active":"2023-10-06T01:39:49.360515+0000","last_became_peered":"2023-10-06T01:39:49.360515+0000","last_unstale":"2023-10-06T01:39:49.362749+0000","last_undegraded":"2023-10-06T01:39:49.362749+0000","last_fullsized":"2023-10-06T01:39:49.362749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:48.332154+0000","last_clean_scrub_stamp":"2023-10-06T01:39:48.332154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:01.629259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.351554+0000","last_change":"2023-10-06T01:39:44.326433+0000","last_active":"2023-10-06T01:39:49.351554+0000","last_peered":"2023-10-06T01:39:49.351554+0000","last_clean":"2023-10-06T01:39:49.351554+0000","last_became_active":"2023-10-06T01:39:44.326167+0000","last_became_peered":"2023-10-06T01:39:44.326167+0000","last_unstale":"2023-10-06T01:39:49.351554+0000","last_undegraded":"2023-10-06T01:39:49.351554+0000","last_fullsized":"2023-10-06T01:39:49.351554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:43.297875+0000","last_clean_scrub_stamp":"2023-10-06T01:39:43.297875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.917652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.342356+0000","last_change":"2023-10-06T01:39:45.330018+0000","last_active":"2023-10-06T01:39:49.342356+0000","last_peered":"2023-10-06T01:39:49.342356+0000","last_clean":"2023-10-06T01:39:49.342356+0000","last_became_active":"2023-10-06T01:39:45.329556+0000","last_became_peered":"2023-10-06T01:39:45.329556+0000","last_unstale":"2023-10-06T01:39:49.342356+0000","last_undegraded":"2023-10-06T01:39:49.342356+0000","last_fullsized":"2023-10-06T01:39:49.342356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343235+0000","last_change":"2023-10-06T01:39:45.333252+0000","last_active":"2023-10-06T01:39:49.343235+0000","last_peered":"2023-10-06T01:39:49.343235+0000","last_clean":"2023-10-06T01:39:49.343235+0000","last_became_active":"2023-10-06T01:39:45.332899+0000","last_became_peered":"2023-10-06T01:39:45.332899+0000","last_unstale":"2023-10-06T01:39:49.343235+0000","last_undegraded":"2023-10-06T01:39:49.343235+0000","last_fullsized":"2023-10-06T01:39:49.343235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:37.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337491+0000","last_change":"2023-10-06T01:39:45.342104+0000","last_active":"2023-10-06T01:39:49.337491+0000","last_peered":"2023-10-06T01:39:49.337491+0000","last_clean":"2023-10-06T01:39:49.337491+0000","last_became_active":"2023-10-06T01:39:45.341825+0000","last_became_peered":"2023-10-06T01:39:45.341825+0000","last_unstale":"2023-10-06T01:39:49.337491+0000","last_undegraded":"2023-10-06T01:39:49.337491+0000","last_fullsized":"2023-10-06T01:39:49.337491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.587945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.340694+0000","last_change":"2023-10-06T01:39:45.329243+0000","last_active":"2023-10-06T01:39:49.340694+0000","last_peered":"2023-10-06T01:39:49.340694+0000","last_clean":"2023-10-06T01:39:49.340694+0000","last_became_active":"2023-10-06T01:39:45.328438+0000","last_became_peered":"2023-10-06T01:39:45.328438+0000","last_unstale":"2023-10-06T01:39:49.340694+0000","last_undegraded":"2023-10-06T01:39:49.340694+0000","last_fullsized":"2023-10-06T01:39:49.340694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.337590+0000","last_change":"2023-10-06T01:39:45.330664+0000","last_active":"2023-10-06T01:39:49.337590+0000","last_peered":"2023-10-06T01:39:49.337590+0000","last_clean":"2023-10-06T01:39:49.337590+0000","last_became_active":"2023-10-06T01:39:45.330394+0000","last_became_peered":"2023-10-06T01:39:45.330394+0000","last_unstale":"2023-10-06T01:39:49.337590+0000","last_undegraded":"2023-10-06T01:39:49.337590+0000","last_fullsized":"2023-10-06T01:39:49.337590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:40.246014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T01:39:49.343700+0000","last_change":"2023-10-06T01:39:45.338933+0000","last_active":"2023-10-06T01:39:49.343700+0000","last_peered":"2023-10-06T01:39:49.343700+0000","last_clean":"2023-10-06T01:39:49.343700+0000","last_became_active":"2023-10-06T01:39:45.338515+0000","last_became_peered":"2023-10-06T01:39:45.338515+0000","last_unstale":"2023-10-06T01:39:49.343700+0000","last_undegraded":"2023-10-06T01:39:49.343700+0000","last_fullsized":"2023-10-06T01:39:49.343700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:44.303359+0000","last_clean_scrub_stamp":"2023-10-06T01:39:44.303359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:39.600509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6619999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8019999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":694806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":103085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":258048,"data_stored":104528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.3090000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.452}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:03.951 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T01:40:03.952 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T01:40:03.952 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T01:40:03.952 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T01:40:04.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.098+0000 7fa59f59e700 1 Processor -- start 2023-10-06T01:40:04.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.098+0000 7fa59f59e700 1 -- start start 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59f59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5a00b2140 0x7fa5a00b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59f59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a00cb7e0 con 0x7fa5a00b3610 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59f59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a00b08a0 con 0x7fa5a00b2140 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00cb190 con 0x7fa5a0159480 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5a00b2140 0x7fa5a00b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.099+0000 7fa59e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:44034/0 (socket says 172.21.15.98:44034) 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59e59c700 1 -- 172.21.15.98:0/999670844 learned_addr learned my addr 172.21.15.98:0/999670844 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5a00b2140 0x7fa5a00b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60486/0 (socket says 172.21.15.98:60486) 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59dd9b700 1 --1- 172.21.15.98:0/999670844 >> v1:172.21.15.184:6789/0 conn(0x7fa5a00b3610 0x7fa5a00cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:39276/0 (socket says 172.21.15.98:39276) 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 -- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 msgr2=0x7fa5a00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 -- 172.21.15.98:0/999670844 >> v1:172.21.15.184:6789/0 conn(0x7fa5a00b3610 legacy=0x7fa5a00cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 -- 172.21.15.98:0/999670844 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00cb470 con 0x7fa5a00b2140 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59d59a700 1 -- 172.21.15.98:0/999670844 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582292751 0 0) 0x7fa5a00cb7e0 con 0x7fa5a00b3610 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59e59c700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:04.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59ed9d700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a00b25c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa598009e60 tx=0x7fa598009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf14c6a75dabf41c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59d59a700 1 -- 172.21.15.98:0/999670844 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59800e070 con 0x7fa5a00b2140 2023-10-06T01:40:04.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59d59a700 1 -- 172.21.15.98:0/999670844 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800bb60 con 0x7fa5a00b2140 2023-10-06T01:40:04.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.100+0000 7fa59d59a700 1 -- 172.21.15.98:0/999670844 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598012430 con 0x7fa5a00b2140 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 -- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 msgr2=0x7fa5a00b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a00b25c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa598009e60 tx=0x7fa598009780 comp rx=0 tx=0).stop 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 -- 172.21.15.98:0/999670844 shutdown_connections 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a00b25c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 --2- 172.21.15.98:0/999670844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a0159480 0x7fa5a00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 -- 172.21.15.98:0/999670844 >> 172.21.15.98:0/999670844 conn(0x7fa5a006ed20 msgr2=0x7fa5a006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 -- 172.21.15.98:0/999670844 shutdown_connections 2023-10-06T01:40:04.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.101+0000 7fa59f59e700 1 -- 172.21.15.98:0/999670844 wait complete. 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.102+0000 7fa59f59e700 1 Processor -- start 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.102+0000 7fa59f59e700 1 -- start start 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a0155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a00b3610 0x7fa5a0147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00cf350 con 0x7fa5a0159480 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00cf630 con 0x7fa5a00b2140 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00cf910 con 0x7fa5a00b3610 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41942/0 (socket says 172.21.15.98:41942) 2023-10-06T01:40:04.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 -- 172.21.15.98:0/1842416976 learned_addr learned my addr 172.21.15.98:0/1842416976 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59dd9b700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a00b3610 0x7fa5a0147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59e59c700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a0155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 -- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a00b3610 msgr2=0x7fa5a0147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a00b3610 0x7fa5a0147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 -- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 msgr2=0x7fa5a0155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a0155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 -- 172.21.15.98:0/1842416976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940094f0 con 0x7fa5a0159480 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59e59c700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a0155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59ed9d700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0069190 tx=0x7fa598019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598027470 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598027610 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.103+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa598009a20 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.104+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980279d0 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.104+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a00cfed0 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.104+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a005f270 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.105+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa598024070 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.105+0000 7fa58e7fc700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.105+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa598087480 con 0x7fa5a0159480 2023-10-06T01:40:04.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.105+0000 7fa59e59c700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.106+0000 7fa59e59c700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa594006d40 tx=0x7fa594008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.110+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa598054200 con 0x7fa5a0159480 2023-10-06T01:40:04.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.333+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa5a00b2b00 con 0x7fa5a0159480 2023-10-06T01:40:04.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.334+0000 7fa58e7fc700 1 -- 172.21.15.98:0/1842416976 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa598057820 con 0x7fa5a0159480 2023-10-06T01:40:04.335 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:04.335 INFO:teuthology.orchestra.run.smithi098.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 msgr2=0x7fa5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa594006d40 tx=0x7fa594008040 comp rx=0 tx=0).stop 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 msgr2=0x7fa5a00ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0069190 tx=0x7fa598019000 comp rx=0 tx=0).stop 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 shutdown_connections 2023-10-06T01:40:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5a0159480 0x7fa5a00ce990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a00b2140 0x7fa5a0155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 --2- 172.21.15.98:0/1842416976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5a00b3610 0x7fa5a0147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 >> 172.21.15.98:0/1842416976 conn(0x7fa5a006ed20 msgr2=0x7fa5a00c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 shutdown_connections 2023-10-06T01:40:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.336+0000 7fa59f59e700 1 -- 172.21.15.98:0/1842416976 wait complete. 2023-10-06T01:40:04.348 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T01:40:04.349 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.462+0000 7fea3d72f700 1 Processor -- start 2023-10-06T01:40:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.462+0000 7fea3d72f700 1 -- start start 2023-10-06T01:40:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3806d550 0x7fea3806d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fea380c8320 0x7fea380878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fea38069560 0x7fea38069980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380c8a20 con 0x7fea38069560 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380c7080 con 0x7fea380c8320 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea3d72f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea38087e30 con 0x7fea3806d550 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea377fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fea38069560 0x7fea38069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea377fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fea38069560 0x7fea38069980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41954/0 (socket says 172.21.15.98:41954) 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea377fe700 1 -- 172.21.15.98:0/2977744110 learned_addr learned my addr 172.21.15.98:0/2977744110 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea367fc700 1 --2- 172.21.15.98:0/2977744110 >> v2:172.21.15.184:3300/0 conn(0x7fea380c8320 0x7fea380878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.463+0000 7fea36ffd700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3806d550 0x7fea3806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 -- 172.21.15.98:0/2977744110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3806d550 msgr2=0x7fea3806d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3806d550 0x7fea3806d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 -- 172.21.15.98:0/2977744110 >> v2:172.21.15.184:3300/0 conn(0x7fea380c8320 msgr2=0x7fea380878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 --2- 172.21.15.98:0/2977744110 >> v2:172.21.15.184:3300/0 conn(0x7fea380c8320 0x7fea380878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 -- 172.21.15.98:0/2977744110 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380c8790 con 0x7fea38069560 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.518+0000 7fea377fe700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea38069560 0x7fea38069980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea3806d180 tx=0x7fea3000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9778e498a5eec625 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea35ffb700 1 -- 172.21.15.98:0/2977744110 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000b2e0 con 0x7fea38069560 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea35ffb700 1 -- 172.21.15.98:0/2977744110 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000b440 con 0x7fea38069560 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea35ffb700 1 -- 172.21.15.98:0/2977744110 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000cbd0 con 0x7fea38069560 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 -- 172.21.15.98:0/2977744110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea38069560 msgr2=0x7fea38069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea38069560 0x7fea38069980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea3806d180 tx=0x7fea3000d040 comp rx=0 tx=0).stop 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 -- 172.21.15.98:0/2977744110 shutdown_connections 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea38069560 0x7fea38069980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 --2- 172.21.15.98:0/2977744110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3806d550 0x7fea3806d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 --2- 172.21.15.98:0/2977744110 >> v2:172.21.15.184:3300/0 conn(0x7fea380c8320 0x7fea380878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 -- 172.21.15.98:0/2977744110 >> 172.21.15.98:0/2977744110 conn(0x7fea380680c0 msgr2=0x7fea380cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 -- 172.21.15.98:0/2977744110 shutdown_connections 2023-10-06T01:40:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.519+0000 7fea3d72f700 1 -- 172.21.15.98:0/2977744110 wait complete. 2023-10-06T01:40:04.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.520+0000 7fea3d72f700 1 Processor -- start 2023-10-06T01:40:04.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 -- start start 2023-10-06T01:40:04.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea38069560 0x7fea38086ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3806d550 0x7fea380b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38167b10 con 0x7fea380c8320 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38166c20 con 0x7fea3806d550 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea3d72f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea38166f00 con 0x7fea38069560 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea377fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea377fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41958/0 (socket says 172.21.15.98:41958) 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea377fe700 1 -- 172.21.15.98:0/3449008823 learned_addr learned my addr 172.21.15.98:0/3449008823 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea367fc700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3806d550 0x7fea380b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.521+0000 7fea36ffd700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea38069560 0x7fea38086ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 -- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea38069560 msgr2=0x7fea38086ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea38069560 0x7fea38086ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 -- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3806d550 msgr2=0x7fea380b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3806d550 0x7fea380b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 -- 172.21.15.98:0/3449008823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3000bb00 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea377fe700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea30001220 tx=0x7fea30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000ae50 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3001d620 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea381671e0 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3001d9e0 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.522+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea381677a0 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.523+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38064ad0 con 0x7fea380c8320 2023-10-06T01:40:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.523+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea3001a070 con 0x7fea380c8320 2023-10-06T01:40:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.524+0000 7fea1effd700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 0x7fea240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.524+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fea30086380 con 0x7fea380c8320 2023-10-06T01:40:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.524+0000 7fea36ffd700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 0x7fea240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.524+0000 7fea36ffd700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 0x7fea240693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea280064e0 tx=0x7fea28009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.528+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea30053220 con 0x7fea380c8320 2023-10-06T01:40:04.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.710+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea38167c50 con 0x7fea380c8320 2023-10-06T01:40:04.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.711+0000 7fea1effd700 1 -- 172.21.15.98:0/3449008823 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fea30056840 con 0x7fea380c8320 2023-10-06T01:40:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:04.712 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:04.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 msgr2=0x7fea240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 0x7fea240693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea280064e0 tx=0x7fea28009190 comp rx=0 tx=0).stop 2023-10-06T01:40:04.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 msgr2=0x7fea380c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea30001220 tx=0x7fea30016000 comp rx=0 tx=0).stop 2023-10-06T01:40:04.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 shutdown_connections 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea24066f30 0x7fea240693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea380c8320 0x7fea380c1fd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea38069560 0x7fea38086ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 --2- 172.21.15.98:0/3449008823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea3806d550 0x7fea380b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 >> 172.21.15.98:0/3449008823 conn(0x7fea380680c0 msgr2=0x7fea380c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 shutdown_connections 2023-10-06T01:40:04.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.713+0000 7fea3d72f700 1 -- 172.21.15.98:0/3449008823 wait complete. 2023-10-06T01:40:04.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:04.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:04.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.844+0000 7f6be66d4700 1 Processor -- start 2023-10-06T01:40:04.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.844+0000 7f6be66d4700 1 -- start start 2023-10-06T01:40:04.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be66d4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6be00ccab0 0x7f6be00d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be66d4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00cdc30 con 0x7f6be01539d0 2023-10-06T01:40:04.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be66d4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00cb500 con 0x7f6be00d0830 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be66d4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00d0c50 con 0x7f6be00ccab0 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be56d2700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6be01539d0 0x7f6be015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43120/0 (socket says 172.21.15.98:43120) 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be56d2700 1 -- 172.21.15.98:0/4217863365 learned_addr learned my addr 172.21.15.98:0/4217863365 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.845+0000 7f6be4ed1700 1 --2- 172.21.15.98:0/4217863365 >> v2:172.21.15.184:3300/0 conn(0x7f6be00ccab0 0x7f6be00d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153084092 0 0) 0x7f6be00cdc30 con 0x7f6be01539d0 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bcc003570 con 0x7f6be01539d0 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814064837 0 0) 0x7f6bcc003570 con 0x7f6be01539d0 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 >> v2:172.21.15.184:3300/0 conn(0x7f6be00ccab0 msgr2=0x7f6be00d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 --2- 172.21.15.98:0/4217863365 >> v2:172.21.15.184:3300/0 conn(0x7f6be00ccab0 0x7f6be00d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 >> v1:172.21.15.184:6789/0 conn(0x7f6be00d0830 legacy=0x7f6be00ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00cd020 con 0x7f6be01539d0 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f6bdc0017f0 con 0x7f6be01539d0 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 >> v1:172.21.15.98:6789/0 conn(0x7f6be01539d0 legacy=0x7f6be015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc003a60 0x7f6bcc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bcc004810 0x7f6bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 0x7f6bcc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00d0c50 con 0x7f6bcc0051c0 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc00c160 con 0x7f6bcc004810 2023-10-06T01:40:04.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc00c320 con 0x7f6bcc003a60 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 0x7f6bcc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be4ed1700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bcc004810 0x7f6bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be56d2700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc003a60 0x7f6bcc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 -- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc003a60 msgr2=0x7f6bcc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc003a60 0x7f6bcc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 -- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bcc004810 msgr2=0x7f6bcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bcc004810 0x7f6bcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 -- 172.21.15.98:0/4217863365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc003570 con 0x7f6bcc0051c0 2023-10-06T01:40:04.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 -- 172.21.15.98:0/4217863365 reap_dead start 2023-10-06T01:40:04.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.846+0000 7f6be5ed3700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 0x7f6bcc00ba60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80018b0 tx=0x7f6bd8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bbde9feb3eaca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8012420 con 0x7f6bcc0051c0 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd80125a0 con 0x7f6bcc0051c0 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6bd7fff700 1 -- 172.21.15.98:0/4217863365 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8012960 con 0x7f6bcc0051c0 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 -- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 msgr2=0x7f6bcc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 0x7f6bcc00ba60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80018b0 tx=0x7f6bd8007d30 comp rx=0 tx=0).stop 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 -- 172.21.15.98:0/4217863365 shutdown_connections 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 --2- 172.21.15.98:0/4217863365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0051c0 0x7f6bcc00ba60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 -- 172.21.15.98:0/4217863365 >> 172.21.15.98:0/4217863365 conn(0x7f6be0068100 msgr2=0x7f6be00684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 -- 172.21.15.98:0/4217863365 shutdown_connections 2023-10-06T01:40:04.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.847+0000 7f6be66d4700 1 -- 172.21.15.98:0/4217863365 wait complete. 2023-10-06T01:40:04.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.848+0000 7f6be66d4700 1 Processor -- start 2023-10-06T01:40:04.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.848+0000 7f6be66d4700 1 -- start start 2023-10-06T01:40:04.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc0051c0 0x7f6be016d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be0141c70 0x7f6be01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00ca2f0 con 0x7f6be0151200 2023-10-06T01:40:04.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be016d480 con 0x7f6be0141c70 2023-10-06T01:40:04.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be66d4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b3c50 con 0x7f6bcc0051c0 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41970/0 (socket says 172.21.15.98:41970) 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 -- 172.21.15.98:0/1184062795 learned_addr learned my addr 172.21.15.98:0/1184062795 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be56d2700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc0051c0 0x7f6be016d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be5ed3700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be0141c70 0x7f6be01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 -- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc0051c0 msgr2=0x7f6be016d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc0051c0 0x7f6be016d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 -- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be0141c70 msgr2=0x7f6be01599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be0141c70 0x7f6be01599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.849+0000 7f6be4ed1700 1 -- 172.21.15.98:0/1184062795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00b3f30 con 0x7f6be0151200 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6be4ed1700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006ac0 tx=0x7f6bd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000f940 con 0x7f6be0151200 2023-10-06T01:40:04.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000faa0 con 0x7f6be0151200 2023-10-06T01:40:04.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be016ce30 con 0x7f6be0151200 2023-10-06T01:40:04.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000fe20 con 0x7f6be0151200 2023-10-06T01:40:04.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.850+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be01420c0 con 0x7f6be0151200 2023-10-06T01:40:04.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.851+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be0142340 con 0x7f6be0151200 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.851+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bd0014650 con 0x7f6be0151200 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.852+0000 7f6bd57fa700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 0x7f6bc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.852+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6bd0018030 con 0x7f6be0151200 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.852+0000 7f6be56d2700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 0x7f6bc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.852+0000 7f6be56d2700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 0x7f6bc40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc002060 tx=0x7f6bdc0021d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:04.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:04.856+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bd0052100 con 0x7f6be0151200 2023-10-06T01:40:05.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.038+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6be016d5f0 con 0x7f6be0151200 2023-10-06T01:40:05.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.038+0000 7f6bd57fa700 1 -- 172.21.15.98:0/1184062795 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6bd0055720 con 0x7f6be0151200 2023-10-06T01:40:05.072 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:05.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 msgr2=0x7f6bc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 0x7f6bc40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc002060 tx=0x7f6bdc0021d0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 msgr2=0x7f6be00c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006ac0 tx=0x7f6bd00091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 shutdown_connections 2023-10-06T01:40:05.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bc4066f30 0x7f6bc40693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6be0141c70 0x7f6be01599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bcc0051c0 0x7f6be016d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 --2- 172.21.15.98:0/1184062795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6be0151200 0x7f6be00c9db0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 >> 172.21.15.98:0/1184062795 conn(0x7f6be0068100 msgr2=0x7f6be00c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 shutdown_connections 2023-10-06T01:40:05.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.040+0000 7f6be66d4700 1 -- 172.21.15.98:0/1184062795 wait complete. 2023-10-06T01:40:05.082 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:05.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.199+0000 7f1a04976700 1 Processor -- start 2023-10-06T01:40:05.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.199+0000 7f1a04976700 1 -- start start 2023-10-06T01:40:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f1a04976700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a00159510 0x7f1a000cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f1a04976700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a001598e0 0x7f1a000cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f1a04976700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a000cf350 con 0x7f1a000ce820 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f1a04976700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000cf610 con 0x7f1a001598e0 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f1a04976700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000cf8f0 con 0x7f1a00159510 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a001598e0 0x7f1a000cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a001598e0 0x7f1a000cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41976/0 (socket says 172.21.15.98:41976) 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 -- 172.21.15.98:0/1355564959 learned_addr learned my addr 172.21.15.98:0/1355564959 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19fe59c700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a00159510 0x7f1a000cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 -- 172.21.15.98:0/1355564959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a00159510 msgr2=0x7f1a000cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:05.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a00159510 0x7f1a000cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 -- 172.21.15.98:0/1355564959 >> v1:172.21.15.184:6789/0 conn(0x7f1a000ce820 legacy=0x7f1a000cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.200+0000 7f19f5d9b700 1 -- 172.21.15.98:0/1355564959 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000cfbd0 con 0x7f1a001598e0 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f19f5d9b700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a000cbae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19e80099b0 tx=0x7f19e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ced969df0c2d02f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f19fdd9b700 1 -- 172.21.15.98:0/1355564959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800b860 con 0x7f1a001598e0 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f19fdd9b700 1 -- 172.21.15.98:0/1355564959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e800b9c0 con 0x7f1a001598e0 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f19fdd9b700 1 -- 172.21.15.98:0/1355564959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800bd60 con 0x7f1a001598e0 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 -- 172.21.15.98:0/1355564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 msgr2=0x7f1a000cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a000cbae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19e80099b0 tx=0x7f19e8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 -- 172.21.15.98:0/1355564959 shutdown_connections 2023-10-06T01:40:05.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a00159510 0x7f1a000cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 --2- 172.21.15.98:0/1355564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a000cbae0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 -- 172.21.15.98:0/1355564959 >> 172.21.15.98:0/1355564959 conn(0x7f1a00089e90 msgr2=0x7f1a000997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 -- 172.21.15.98:0/1355564959 shutdown_connections 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.201+0000 7f1a04976700 1 -- 172.21.15.98:0/1355564959 wait complete. 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 Processor -- start 2023-10-06T01:40:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 -- start start 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a000ce820 0x7f1a0015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a00159510 0x7f1a0016d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0016e3e0 con 0x7f1a001598e0 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b6340 con 0x7f1a00159510 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f1a04976700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0016e100 con 0x7f1a000ce820 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f19fed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f19fed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41990/0 (socket says 172.21.15.98:41990) 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.203+0000 7f19fed9d700 1 -- 172.21.15.98:0/145728400 learned_addr learned my addr 172.21.15.98:0/145728400 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19f5d9b700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a00159510 0x7f1a0016d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fe59c700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a000ce820 0x7f1a0015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 -- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a000ce820 msgr2=0x7f1a0015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a000ce820 0x7f1a0015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 -- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a00159510 msgr2=0x7f1a0016d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a00159510 0x7f1a0016d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 -- 172.21.15.98:0/145728400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80094f0 con 0x7f1a001598e0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19fed9d700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c8e0 tx=0x7f19f8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ce80 con 0x7f1a001598e0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800a7e0 con 0x7f1a001598e0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0016a0e0 con 0x7f1a001598e0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ab80 con 0x7f1a001598e0 2023-10-06T01:40:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.204+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0016a6a0 con 0x7f1a001598e0 2023-10-06T01:40:05.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.205+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00169f40 con 0x7f1a001598e0 2023-10-06T01:40:05.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.205+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19f8012420 con 0x7f1a001598e0 2023-10-06T01:40:05.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.206+0000 7f19f6ffd700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 0x7f19e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.206+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f19f8084cd0 con 0x7f1a001598e0 2023-10-06T01:40:05.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.206+0000 7f19fe59c700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 0x7f19e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.206+0000 7f19fe59c700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 0x7f19e00693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006fd0 tx=0x7f19f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.210+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a00169f40 con 0x7f1a001598e0 2023-10-06T01:40:05.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.396+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a00169f40 con 0x7f1a001598e0 2023-10-06T01:40:05.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.397+0000 7f19f6ffd700 1 -- 172.21.15.98:0/145728400 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a00169f40 con 0x7f1a001598e0 2023-10-06T01:40:05.398 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 msgr2=0x7f19e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 0x7f19e00693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006fd0 tx=0x7f19f0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 msgr2=0x7f1a0016da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c8e0 tx=0x7f19f8001710 comp rx=0 tx=0).stop 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 shutdown_connections 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f19e0066f30 0x7f19e00693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a001598e0 0x7f1a0016da80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a000ce820 0x7f1a0015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 --2- 172.21.15.98:0/145728400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a00159510 0x7f1a0016d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 >> 172.21.15.98:0/145728400 conn(0x7f1a00089e90 msgr2=0x7f1a0014b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 shutdown_connections 2023-10-06T01:40:05.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.399+0000 7f1a04976700 1 -- 172.21.15.98:0/145728400 wait complete. 2023-10-06T01:40:05.411 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:05.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.527+0000 7fb06834e700 1 Processor -- start 2023-10-06T01:40:05.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.527+0000 7fb06834e700 1 -- start start 2023-10-06T01:40:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb06834e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb060140d00 0x7fb0600c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb06834e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0600c6c60 0x7fb06006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb06834e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0601410d0 con 0x7fb06006d920 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb06834e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0600c7ea0 con 0x7fb060140d00 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb06834e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06006f720 con 0x7fb0600c6c60 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb0660ea700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb060140d00 0x7fb0600c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb0660ea700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb060140d00 0x7fb0600c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42002/0 (socket says 172.21.15.98:42002) 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.528+0000 7fb0660ea700 1 -- 172.21.15.98:0/4248038061 learned_addr learned my addr 172.21.15.98:0/4248038061 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0658e9700 1 --2- 172.21.15.98:0/4248038061 >> v2:172.21.15.184:3300/0 conn(0x7fb0600c6c60 0x7fb06006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0660ea700 1 -- 172.21.15.98:0/4248038061 >> v2:172.21.15.184:3300/0 conn(0x7fb0600c6c60 msgr2=0x7fb06006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0660ea700 1 --2- 172.21.15.98:0/4248038061 >> v2:172.21.15.184:3300/0 conn(0x7fb0600c6c60 0x7fb06006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0660ea700 1 -- 172.21.15.98:0/4248038061 >> v1:172.21.15.184:6789/0 conn(0x7fb06006d920 legacy=0x7fb06015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0660ea700 1 -- 172.21.15.98:0/4248038061 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06006dd20 con 0x7fb060140d00 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0660ea700 1 --2- 172.21.15.98:0/4248038061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb060140d00 0x7fb0600c6720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0600b20e0 tx=0x7fb05c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af833feee5c3aae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0650e8700 1 -- 172.21.15.98:0/4248038061 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00e070 con 0x7fb060140d00 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0650e8700 1 -- 172.21.15.98:0/4248038061 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00ba10 con 0x7fb060140d00 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb0650e8700 1 -- 172.21.15.98:0/4248038061 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0092e0 con 0x7fb060140d00 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb06834e700 1 -- 172.21.15.98:0/4248038061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb060140d00 msgr2=0x7fb0600c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.529+0000 7fb06834e700 1 --2- 172.21.15.98:0/4248038061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb060140d00 0x7fb0600c6720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0600b20e0 tx=0x7fb05c009840 comp rx=0 tx=0).stop 2023-10-06T01:40:05.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 -- 172.21.15.98:0/4248038061 shutdown_connections 2023-10-06T01:40:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 --2- 172.21.15.98:0/4248038061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb060140d00 0x7fb0600c6720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 --2- 172.21.15.98:0/4248038061 >> v2:172.21.15.184:3300/0 conn(0x7fb0600c6c60 0x7fb06006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 -- 172.21.15.98:0/4248038061 >> 172.21.15.98:0/4248038061 conn(0x7fb0600cc0e0 msgr2=0x7fb0600cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 -- 172.21.15.98:0/4248038061 shutdown_connections 2023-10-06T01:40:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.530+0000 7fb06834e700 1 -- 172.21.15.98:0/4248038061 wait complete. 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 Processor -- start 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 -- start start 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 0x7fb0601687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 0x7fb060138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600c3190 con 0x7fb0600c6c60 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600c3d10 con 0x7fb060140d00 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb06834e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600c33e0 con 0x7fb06006d920 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb0658e9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb0658e9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42010/0 (socket says 172.21.15.98:42010) 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.531+0000 7fb0658e9700 1 -- 172.21.15.98:0/3102232077 learned_addr learned my addr 172.21.15.98:0/3102232077 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0660ea700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 0x7fb0601687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0668eb700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 0x7fb060138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 -- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 msgr2=0x7fb0601687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 0x7fb0601687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 -- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 msgr2=0x7fb060138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 0x7fb060138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 -- 172.21.15.98:0/3102232077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009ae0 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0668eb700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 0x7fb060138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0660ea700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 0x7fb0601687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0658e9700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d8c0 tx=0x7fb050008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000de60 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05000b780 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0600c36c0 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000bb20 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.532+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb060138f80 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.533+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb060064ad0 con 0x7fb0600c6c60 2023-10-06T01:40:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.533+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb05001e460 con 0x7fb0600c6c60 2023-10-06T01:40:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.534+0000 7fb0567fc700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 0x7fb0440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.534+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb050085030 con 0x7fb0600c6c60 2023-10-06T01:40:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.534+0000 7fb0660ea700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 0x7fb0440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.534+0000 7fb0660ea700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 0x7fb0440693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009880 tx=0x7fb05c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.538+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb050051e50 con 0x7fb0600c6c60 2023-10-06T01:40:05.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.719+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb06004bb80 con 0x7fb0600c6c60 2023-10-06T01:40:05.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.719+0000 7fb0567fc700 1 -- 172.21.15.98:0/3102232077 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb050055470 con 0x7fb0600c6c60 2023-10-06T01:40:05.720 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 msgr2=0x7fb0440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 0x7fb0440693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009880 tx=0x7fb05c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 msgr2=0x7fb0600c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d8c0 tx=0x7fb050008040 comp rx=0 tx=0).stop 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 shutdown_connections 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb044066f30 0x7fb0440693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb060140d00 0x7fb060138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb06006d920 0x7fb0601687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 --2- 172.21.15.98:0/3102232077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0600c6c60 0x7fb0600c41a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 >> 172.21.15.98:0/3102232077 conn(0x7fb0600cc0e0 msgr2=0x7fb06013ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 shutdown_connections 2023-10-06T01:40:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.721+0000 7fb06834e700 1 -- 172.21.15.98:0/3102232077 wait complete. 2023-10-06T01:40:05.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:05.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.850+0000 7f5a5ddb3700 1 Processor -- start 2023-10-06T01:40:05.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.850+0000 7f5a5ddb3700 1 -- start start 2023-10-06T01:40:05.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a5ddb3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a5ddb3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a580675f0 0x7f5a58067a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a5ddb3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a580cbfb0 con 0x7f5a5806eff0 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a5ddb3700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a58159c00 con 0x7f5a580675f0 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a5ddb3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5806f460 con 0x7f5a580cc860 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a57fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a580675f0 0x7f5a58067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a57fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a580675f0 0x7f5a58067a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42026/0 (socket says 172.21.15.98:42026) 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a57fff700 1 -- 172.21.15.98:0/1647741077 learned_addr learned my addr 172.21.15.98:0/1647741077 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.851+0000 7f5a577fe700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a57fff700 1 -- 172.21.15.98:0/1647741077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 msgr2=0x7f5a580ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a57fff700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a57fff700 1 -- 172.21.15.98:0/1647741077 >> v1:172.21.15.184:6789/0 conn(0x7f5a5806eff0 legacy=0x7f5a580c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a57fff700 1 -- 172.21.15.98:0/1647741077 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58159df0 con 0x7f5a580675f0 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a57fff700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a580675f0 0x7f5a58067a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a500099b0 tx=0x7f5a50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e126c84bedb84168 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a567fc700 1 -- 172.21.15.98:0/1647741077 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50009ea0 con 0x7f5a580675f0 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a567fc700 1 -- 172.21.15.98:0/1647741077 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5000b860 con 0x7f5a580675f0 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a567fc700 1 -- 172.21.15.98:0/1647741077 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5000bc00 con 0x7f5a580675f0 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/1647741077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a580675f0 msgr2=0x7f5a58067a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.852+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a580675f0 0x7f5a58067a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a500099b0 tx=0x7f5a50009250 comp rx=0 tx=0).stop 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/1647741077 shutdown_connections 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a580675f0 0x7f5a58067a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/1647741077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/1647741077 >> 172.21.15.98:0/1647741077 conn(0x7f5a580691d0 msgr2=0x7f5a580cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/1647741077 shutdown_connections 2023-10-06T01:40:05.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.853+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/1647741077 wait complete. 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 Processor -- start 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 -- start start 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a580675f0 0x7f5a5815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.854+0000 7f5a5ddb3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5815c2c0 con 0x7f5a5806eff0 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a5ddb3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581578a0 con 0x7f5a580675f0 2023-10-06T01:40:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a5ddb3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a58157b10 con 0x7f5a580cc860 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42030/0 (socket says 172.21.15.98:42030) 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 -- 172.21.15.98:0/3939498071 learned_addr learned my addr 172.21.15.98:0/3939498071 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a57fff700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a577fe700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a580675f0 0x7f5a5815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 -- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 msgr2=0x7f5a580c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 -- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a580675f0 msgr2=0x7f5a5815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a580675f0 0x7f5a5815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 -- 172.21.15.98:0/3939498071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500094f0 con 0x7f5a5806eff0 2023-10-06T01:40:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a57fff700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a56ffd700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c590 tx=0x7f5a4c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011590 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0116f0 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a58157df0 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011ab0 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.855+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a58158750 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.856+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a58064ad0 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.857+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a4c01b3b0 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.857+0000 7f5a3f7fe700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 0x7f5a440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.857+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5a4c083f30 con 0x7f5a5806eff0 2023-10-06T01:40:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.857+0000 7f5a577fe700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 0x7f5a440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.857+0000 7f5a577fe700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 0x7f5a440693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a48007620 tx=0x7f5a4800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:05.861+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a4c050d50 con 0x7f5a5806eff0 2023-10-06T01:40:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.044+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a5804bb80 con 0x7f5a5806eff0 2023-10-06T01:40:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.044+0000 7f5a3f7fe700 1 -- 172.21.15.98:0/3939498071 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a5806eff0 2023-10-06T01:40:06.045 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 msgr2=0x7f5a440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 0x7f5a440693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a48007620 tx=0x7f5a4800b040 comp rx=0 tx=0).stop 2023-10-06T01:40:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 msgr2=0x7f5a58158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c590 tx=0x7f5a4c00c950 comp rx=0 tx=0).stop 2023-10-06T01:40:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 shutdown_connections 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5a44066f30 0x7f5a440693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a580cc860 0x7f5a580c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a580675f0 0x7f5a5815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 --2- 172.21.15.98:0/3939498071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a5806eff0 0x7f5a58158210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 >> 172.21.15.98:0/3939498071 conn(0x7f5a580691d0 msgr2=0x7f5a58156c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 shutdown_connections 2023-10-06T01:40:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.046+0000 7f5a5ddb3700 1 -- 172.21.15.98:0/3939498071 wait complete. 2023-10-06T01:40:06.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:06.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.176+0000 7f1139b9f700 1 Processor -- start 2023-10-06T01:40:06.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.176+0000 7f1139b9f700 1 -- start start 2023-10-06T01:40:06.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.177+0000 7f1139b9f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.177+0000 7f1139b9f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1134067b30 0x7f11340b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.177+0000 7f1139b9f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11340b0310 con 0x7f11340afde0 2023-10-06T01:40:06.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.177+0000 7f1139b9f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11340cdea0 con 0x7f1134067b30 2023-10-06T01:40:06.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.177+0000 7f1139b9f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1134067fd0 con 0x7f11341593c0 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1134067b30 0x7f11340b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f1138b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f1138b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:44130/0 (socket says 172.21.15.98:44130) 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1134067b30 0x7f11340b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:60564/0 (socket says 172.21.15.98:60564) 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f1138b9d700 1 -- 172.21.15.98:0/3100121910 learned_addr learned my addr 172.21.15.98:0/3100121910 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 -- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 msgr2=0x7f11340675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 -- 172.21.15.98:0/3100121910 >> v1:172.21.15.184:6789/0 conn(0x7f11340afde0 legacy=0x7f11340ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f11337fe700 1 -- 172.21.15.98:0/3100121910 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810373314 0 0) 0x7f11340b0310 con 0x7f11340afde0 2023-10-06T01:40:06.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 -- 172.21.15.98:0/3100121910 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340c9040 con 0x7f1134067b30 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f1138b9d700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f113939e700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340b2970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f112c009e60 tx=0x7f112c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df554b1134264ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f11337fe700 1 -- 172.21.15.98:0/3100121910 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00e070 con 0x7f1134067b30 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f11337fe700 1 -- 172.21.15.98:0/3100121910 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00bb60 con 0x7f1134067b30 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.178+0000 7f11337fe700 1 -- 172.21.15.98:0/3100121910 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c012430 con 0x7f1134067b30 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 -- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 msgr2=0x7f11340b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340b2970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f112c009e60 tx=0x7f112c009780 comp rx=0 tx=0).stop 2023-10-06T01:40:06.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 -- 172.21.15.98:0/3100121910 shutdown_connections 2023-10-06T01:40:06.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340b2970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 --2- 172.21.15.98:0/3100121910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 -- 172.21.15.98:0/3100121910 >> 172.21.15.98:0/3100121910 conn(0x7f11340691d0 msgr2=0x7f11340cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 -- 172.21.15.98:0/3100121910 shutdown_connections 2023-10-06T01:40:06.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.179+0000 7f1139b9f700 1 -- 172.21.15.98:0/3100121910 wait complete. 2023-10-06T01:40:06.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.180+0000 7f1139b9f700 1 Processor -- start 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.180+0000 7f1139b9f700 1 -- start start 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340ca990 con 0x7f11340afde0 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340ca0a0 con 0x7f1134067b30 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1139b9f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340ca340 con 0x7f11341593c0 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42038/0 (socket says 172.21.15.98:42038) 2023-10-06T01:40:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 -- 172.21.15.98:0/2238869505 learned_addr learned my addr 172.21.15.98:0/2238869505 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1138b9d700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f113939e700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 -- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 msgr2=0x7f11340cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 -- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 msgr2=0x7f11340cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.181+0000 7f1133fff700 1 -- 172.21.15.98:0/2238869505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240094f0 con 0x7f11340afde0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1133fff700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f112800c7e0 tx=0x7f1128007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280105c0 con 0x7f11340afde0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128010740 con 0x7f11340afde0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c009a20 con 0x7f11340afde0 2023-10-06T01:40:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128010b00 con 0x7f11340afde0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.182+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340989c0 con 0x7f11340afde0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.183+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113405f270 con 0x7f11340afde0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.183+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f112801b3b0 con 0x7f11340afde0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.183+0000 7f1130ff9700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.183+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1128083fd0 con 0x7f11340afde0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.183+0000 7f1138b9d700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.184+0000 7f1138b9d700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1134098270 tx=0x7f1124009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.187+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1128050d50 con 0x7f11340afde0 2023-10-06T01:40:06.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.367+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f11340caad0 con 0x7f11340afde0 2023-10-06T01:40:06.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.368+0000 7f1130ff9700 1 -- 172.21.15.98:0/2238869505 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1128017070 con 0x7f11340afde0 2023-10-06T01:40:06.368 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:06.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 msgr2=0x7f11200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1134098270 tx=0x7f1124009250 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 msgr2=0x7f1134098480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f112800c7e0 tx=0x7f1128007df0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 shutdown_connections 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11341593c0 0x7f11340cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1134067b30 0x7f11340cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 --2- 172.21.15.98:0/2238869505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340afde0 0x7f1134098480 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.369+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 >> 172.21.15.98:0/2238869505 conn(0x7f11340691d0 msgr2=0x7f11340ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.370+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 shutdown_connections 2023-10-06T01:40:06.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.370+0000 7f1139b9f700 1 -- 172.21.15.98:0/2238869505 wait complete. 2023-10-06T01:40:06.382 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:06.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.495+0000 7f87386f8700 1 Processor -- start 2023-10-06T01:40:06.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.495+0000 7f87386f8700 1 -- start start 2023-10-06T01:40:06.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f87386f8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 0x7f87300cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f87386f8700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300ced30 con 0x7f873015db50 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f87386f8700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300cf010 con 0x7f87300ad620 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f87386f8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87300ccad0 con 0x7f87300ad250 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f8736c95700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f873015db50 0x7f87300ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43134/0 (socket says 172.21.15.98:43134) 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.496+0000 7f8736c95700 1 -- 172.21.15.98:0/288822478 learned_addr learned my addr 172.21.15.98:0/288822478 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8736494700 1 --2- 172.21.15.98:0/288822478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 0x7f87300cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331873925 0 0) 0x7f87300ced30 con 0x7f873015db50 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c003570 con 0x7f873015db50 2023-10-06T01:40:06.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252703849 0 0) 0x7f87300cf010 con 0x7f87300ad620 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87300ced30 con 0x7f87300ad620 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766755549 0 0) 0x7f871c003570 con 0x7f873015db50 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8736494700 1 -- 172.21.15.98:0/288822478 >> v1:172.21.15.184:6789/0 conn(0x7f87300ad620 legacy=0x7f873015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8736494700 1 -- 172.21.15.98:0/288822478 >> v1:172.21.15.98:6789/0 conn(0x7f873015db50 legacy=0x7f87300ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8736494700 1 -- 172.21.15.98:0/288822478 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300ccdb0 con 0x7f87300ad250 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8736494700 1 --2- 172.21.15.98:0/288822478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 0x7f87300cd2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f872c009ee0 tx=0x7f872c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ea19f88ef5c8fdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00e070 con 0x7f87300ad250 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00b870 con 0x7f87300ad250 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.497+0000 7f8735492700 1 -- 172.21.15.98:0/288822478 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728001470 con 0x7f87300ad250 2023-10-06T01:40:06.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 -- 172.21.15.98:0/288822478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 msgr2=0x7f87300cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 --2- 172.21.15.98:0/288822478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 0x7f87300cd2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f872c009ee0 tx=0x7f872c009780 comp rx=0 tx=0).stop 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 -- 172.21.15.98:0/288822478 shutdown_connections 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 --2- 172.21.15.98:0/288822478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87300ad250 0x7f87300cd2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 -- 172.21.15.98:0/288822478 >> 172.21.15.98:0/288822478 conn(0x7f8730067dd0 msgr2=0x7f87300ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 -- 172.21.15.98:0/288822478 shutdown_connections 2023-10-06T01:40:06.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.498+0000 7f87386f8700 1 -- 172.21.15.98:0/288822478 wait complete. 2023-10-06T01:40:06.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.499+0000 7f87386f8700 1 Processor -- start 2023-10-06T01:40:06.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.499+0000 7f87386f8700 1 -- start start 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 0x7f87300b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 0x7f87300b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300c7cb0 con 0x7f87300ad250 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300add50 con 0x7f87300ad620 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87300c6e00 con 0x7f873015db50 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42050/0 (socket says 172.21.15.98:42050) 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 -- 172.21.15.98:0/1601829525 learned_addr learned my addr 172.21.15.98:0/1601829525 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736c95700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 0x7f87300b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8735c93700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 0x7f87300b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 -- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 msgr2=0x7f87300b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 0x7f87300b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 -- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 msgr2=0x7f87300b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 0x7f87300b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 -- 172.21.15.98:0/1601829525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c009a20 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8735c93700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 0x7f87300b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736c95700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 0x7f87300b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f8736494700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b610 tx=0x7f872c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c013a30 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c013bb0 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87300c70a0 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c026400 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.500+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87300c7660 con 0x7f87300ad250 2023-10-06T01:40:06.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.501+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8730064ad0 con 0x7f87300ad250 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.502+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f872c019030 con 0x7f87300ad250 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.502+0000 7f87267fc700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.502+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f872c0866f0 con 0x7f87300ad250 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.502+0000 7f8735c93700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.503+0000 7f8735c93700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 0x7f87140693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f873016d420 tx=0x7f8720009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.506+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f872c053510 con 0x7f87300ad250 2023-10-06T01:40:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.730+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87300cc7a0 con 0x7f87300ad250 2023-10-06T01:40:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.731+0000 7f87267fc700 1 -- 172.21.15.98:0/1601829525 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f872c056b30 con 0x7f87300ad250 2023-10-06T01:40:06.731 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:06.731 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 msgr2=0x7f87140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 0x7f87140693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f873016d420 tx=0x7f8720009040 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 msgr2=0x7f87300b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b610 tx=0x7f872c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 shutdown_connections 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f873015db50 0x7f87300b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ad250 0x7f87300b1550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 --2- 172.21.15.98:0/1601829525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87300ad620 0x7f87300b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 >> 172.21.15.98:0/1601829525 conn(0x7f8730067dd0 msgr2=0x7f873015c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 shutdown_connections 2023-10-06T01:40:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.733+0000 7f87386f8700 1 -- 172.21.15.98:0/1601829525 wait complete. 2023-10-06T01:40:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:06.746 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:06.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.859+0000 7efd62a69700 1 Processor -- start 2023-10-06T01:40:06.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.859+0000 7efd62a69700 1 -- start start 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62a69700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd5c0cd370 0x7efd5c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62a69700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd5c0c6260 0x7efd5c0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62a69700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0cc840 con 0x7efd5c1594b0 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62a69700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0cbc70 con 0x7efd5c0cd370 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62a69700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0cf440 con 0x7efd5c0c6260 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd5c0cd370 0x7efd5c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd5c0cd370 0x7efd5c0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42060/0 (socket says 172.21.15.98:42060) 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 -- 172.21.15.98:0/564665576 learned_addr learned my addr 172.21.15.98:0/564665576 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd62268700 1 --2- 172.21.15.98:0/564665576 >> v2:172.21.15.184:3300/0 conn(0x7efd5c0c6260 0x7efd5c0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 -- 172.21.15.98:0/564665576 >> v2:172.21.15.184:3300/0 conn(0x7efd5c0c6260 msgr2=0x7efd5c0cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 --2- 172.21.15.98:0/564665576 >> v2:172.21.15.184:3300/0 conn(0x7efd5c0c6260 0x7efd5c0cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 -- 172.21.15.98:0/564665576 >> v1:172.21.15.184:6789/0 conn(0x7efd5c1594b0 legacy=0x7efd5c0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.860+0000 7efd61266700 1 -- 172.21.15.98:0/564665576 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0cbf10 con 0x7efd5c0cd370 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd61266700 1 --2- 172.21.15.98:0/564665576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0ce9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca910 tx=0x7efd4c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec1cba78f3d3e5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd60a65700 1 -- 172.21.15.98:0/564665576 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c005c10 con 0x7efd5c0cd370 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd60a65700 1 -- 172.21.15.98:0/564665576 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c005d70 con 0x7efd5c0cd370 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd60a65700 1 -- 172.21.15.98:0/564665576 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00b8a0 con 0x7efd5c0cd370 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 -- 172.21.15.98:0/564665576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 msgr2=0x7efd5c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 --2- 172.21.15.98:0/564665576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0ce9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca910 tx=0x7efd4c009220 comp rx=0 tx=0).stop 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 -- 172.21.15.98:0/564665576 shutdown_connections 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 --2- 172.21.15.98:0/564665576 >> v2:172.21.15.184:3300/0 conn(0x7efd5c0c6260 0x7efd5c0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 --2- 172.21.15.98:0/564665576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0ce9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 -- 172.21.15.98:0/564665576 >> 172.21.15.98:0/564665576 conn(0x7efd5c099600 msgr2=0x7efd5c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 -- 172.21.15.98:0/564665576 shutdown_connections 2023-10-06T01:40:06.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.861+0000 7efd62a69700 1 -- 172.21.15.98:0/564665576 wait complete. 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.862+0000 7efd62a69700 1 Processor -- start 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.862+0000 7efd62a69700 1 -- start start 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd5c0c6260 0x7efd5c1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd5c1594b0 0x7efd5c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c9f30 con 0x7efd5c0cd370 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c1525e0 con 0x7efd5c0c6260 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62a69700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c9c50 con 0x7efd5c1594b0 2023-10-06T01:40:06.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd61266700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd61266700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42076/0 (socket says 172.21.15.98:42076) 2023-10-06T01:40:06.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd61266700 1 -- 172.21.15.98:0/4240664692 learned_addr learned my addr 172.21.15.98:0/4240664692 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:06.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd62268700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd5c1594b0 0x7efd5c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.863+0000 7efd61a67700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd5c0c6260 0x7efd5c1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 -- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd5c1594b0 msgr2=0x7efd5c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd5c1594b0 0x7efd5c0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 -- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd5c0c6260 msgr2=0x7efd5c1520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd5c0c6260 0x7efd5c1520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 -- 172.21.15.98:0/4240664692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009480 con 0x7efd5c0cd370 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd61266700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06fc60 tx=0x7efd4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c01a420 con 0x7efd5c0cd370 2023-10-06T01:40:06.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c01a5e0 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c16d0b0 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c01a9a0 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.864+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c16d670 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.865+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c16d890 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.865+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd4c01ad10 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.866+0000 7efd51ffb700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 0x7efd400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.866+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7efd4c085b90 con 0x7efd5c0cd370 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.866+0000 7efd61a67700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 0x7efd400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:06.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.866+0000 7efd61a67700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 0x7efd400693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd58003a20 tx=0x7efd58007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:06.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:06.870+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd4c052a60 con 0x7efd5c0cd370 2023-10-06T01:40:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.054+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd5c05f270 con 0x7efd5c0cd370 2023-10-06T01:40:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.055+0000 7efd51ffb700 1 -- 172.21.15.98:0/4240664692 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7efd4c029020 con 0x7efd5c0cd370 2023-10-06T01:40:07.056 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:07.056 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:07.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 msgr2=0x7efd400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 0x7efd400693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd58003a20 tx=0x7efd58007f10 comp rx=0 tx=0).stop 2023-10-06T01:40:07.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 msgr2=0x7efd5c0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06fc60 tx=0x7efd4c016000 comp rx=0 tx=0).stop 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 shutdown_connections 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd40066f30 0x7efd400693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd5c1594b0 0x7efd5c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd5c0c6260 0x7efd5c1520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 --2- 172.21.15.98:0/4240664692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c0cd370 0x7efd5c0c9400 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.057+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 >> 172.21.15.98:0/4240664692 conn(0x7efd5c099600 msgr2=0x7efd5c069780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.058+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 shutdown_connections 2023-10-06T01:40:07.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.058+0000 7efd62a69700 1 -- 172.21.15.98:0/4240664692 wait complete. 2023-10-06T01:40:07.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:07.071 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:07.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.188+0000 7f546671c700 1 Processor -- start 2023-10-06T01:40:07.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.188+0000 7f546671c700 1 -- start start 2023-10-06T01:40:07.194 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f546671c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5460159510 0x7f54600cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f546671c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f54600cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f546671c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54600cf350 con 0x7f54600ce820 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f546671c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600cf610 con 0x7f5460159510 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f546671c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600cf8f0 con 0x7f54601598e0 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f5464cb9700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f54600ce820 0x7f54600cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43142/0 (socket says 172.21.15.98:43142) 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f5464cb9700 1 -- 172.21.15.98:0/3894269305 learned_addr learned my addr 172.21.15.98:0/3894269305 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f545ffff700 1 --2- 172.21.15.98:0/3894269305 >> v2:172.21.15.184:3300/0 conn(0x7f5460159510 0x7f54600cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.195 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f545f7fe700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f54600cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.189+0000 7f545effd700 1 -- 172.21.15.98:0/3894269305 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188479178 0 0) 0x7f54600cf350 con 0x7f54600ce820 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545effd700 1 -- 172.21.15.98:0/3894269305 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5448003570 con 0x7f54600ce820 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545ffff700 1 -- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 msgr2=0x7f54600cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545ffff700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f54600cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545ffff700 1 -- 172.21.15.98:0/3894269305 >> v1:172.21.15.98:6789/0 conn(0x7f54600ce820 legacy=0x7f54600cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545ffff700 1 -- 172.21.15.98:0/3894269305 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600cfbd0 con 0x7f5460159510 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545ffff700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600cb5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54500098e0 tx=0x7f54500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5c8979371c0b1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545effd700 1 -- 172.21.15.98:0/3894269305 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011630 con 0x7f5460159510 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545effd700 1 -- 172.21.15.98:0/3894269305 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450011790 con 0x7f5460159510 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f545effd700 1 -- 172.21.15.98:0/3894269305 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011b30 con 0x7f5460159510 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f546671c700 1 -- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 msgr2=0x7f54600cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.190+0000 7f546671c700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600cb5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54500098e0 tx=0x7f54500091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 -- 172.21.15.98:0/3894269305 shutdown_connections 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600cb5a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 --2- 172.21.15.98:0/3894269305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f54600cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 -- 172.21.15.98:0/3894269305 >> 172.21.15.98:0/3894269305 conn(0x7f5460089e90 msgr2=0x7f54600997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 -- 172.21.15.98:0/3894269305 shutdown_connections 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.191+0000 7f546671c700 1 -- 172.21.15.98:0/3894269305 wait complete. 2023-10-06T01:40:07.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 Processor -- start 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 -- start start 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f546015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5460169e80 con 0x7f54600ce820 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b4600 con 0x7f5460159510 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.192+0000 7f546671c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b48e0 con 0x7f54601598e0 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42088/0 (socket says 172.21.15.98:42088) 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 -- 172.21.15.98:0/1802989550 learned_addr learned my addr 172.21.15.98:0/1802989550 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:07.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f5464cb9700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f546015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545f7fe700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 -- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 msgr2=0x7f546015cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f546015cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 -- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 msgr2=0x7f54600b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 -- 172.21.15.98:0/1802989550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450009420 con 0x7f54600ce820 2023-10-06T01:40:07.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f545ffff700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5460067c60 tx=0x7f5450017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011900 con 0x7f54600ce820 2023-10-06T01:40:07.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450011a60 con 0x7f54600ce820 2023-10-06T01:40:07.200 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54600b4bc0 con 0x7f54600ce820 2023-10-06T01:40:07.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450025640 con 0x7f54600ce820 2023-10-06T01:40:07.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.193+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600b41f0 con 0x7f54600ce820 2023-10-06T01:40:07.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.194+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5460064ad0 con 0x7f54600ce820 2023-10-06T01:40:07.202 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.194+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5450025a30 con 0x7f54600ce820 2023-10-06T01:40:07.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.195+0000 7f5447fff700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 0x7f544c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.195+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5450086050 con 0x7f54600ce820 2023-10-06T01:40:07.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.195+0000 7f545f7fe700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 0x7f544c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.195+0000 7f545f7fe700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 0x7f544c069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5460064f20 tx=0x7f54540091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.199+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5450052e70 con 0x7f54600ce820 2023-10-06T01:40:07.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.382+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f54600cfeb0 con 0x7f54600ce820 2023-10-06T01:40:07.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.382+0000 7f5447fff700 1 -- 172.21.15.98:0/1802989550 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5450056490 con 0x7f54600ce820 2023-10-06T01:40:07.383 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 msgr2=0x7f544c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 0x7f544c069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5460064f20 tx=0x7f54540091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 msgr2=0x7f546014cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5460067c60 tx=0x7f5450017000 comp rx=0 tx=0).stop 2023-10-06T01:40:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 shutdown_connections 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f544c066f60 0x7f544c069410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54601598e0 0x7f546015cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54600ce820 0x7f546014cb40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 --2- 172.21.15.98:0/1802989550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5460159510 0x7f54600b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 >> 172.21.15.98:0/1802989550 conn(0x7f5460089e90 msgr2=0x7f546014f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 shutdown_connections 2023-10-06T01:40:07.386 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.384+0000 7f546671c700 1 -- 172.21.15.98:0/1802989550 wait complete. 2023-10-06T01:40:07.397 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:07.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.513+0000 7f050b7fc700 1 Processor -- start 2023-10-06T01:40:07.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.513+0000 7f050b7fc700 1 -- start start 2023-10-06T01:40:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f050b7fc700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0504159510 0x7f05040cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f050b7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f050b7fc700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05040cf350 con 0x7f05040ce820 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f050b7fc700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05040cf610 con 0x7f0504159510 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f050b7fc700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05040cf8f0 con 0x7f05041598e0 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f0508d97700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f0509598700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0504159510 0x7f05040cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f0508d97700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:44188/0 (socket says 172.21.15.98:44188) 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.514+0000 7f0508d97700 1 -- 172.21.15.98:0/402166732 learned_addr learned my addr 172.21.15.98:0/402166732 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0509598700 1 -- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 msgr2=0x7f05040cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0509598700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0509598700 1 -- 172.21.15.98:0/402166732 >> v1:172.21.15.184:6789/0 conn(0x7f05040ce820 legacy=0x7f05040cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0509598700 1 -- 172.21.15.98:0/402166732 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05040cfbd0 con 0x7f0504159510 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f04fbfff700 1 -- 172.21.15.98:0/402166732 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194553839 0 0) 0x7f05040cf350 con 0x7f05040ce820 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0508d97700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f0509598700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0504159510 0x7f05040cb5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0500009e60 tx=0x7f0500009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57822f8e69f90b0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f04fbfff700 1 -- 172.21.15.98:0/402166732 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050000e070 con 0x7f0504159510 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f04fbfff700 1 -- 172.21.15.98:0/402166732 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050000bb60 con 0x7f0504159510 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.515+0000 7f04fbfff700 1 -- 172.21.15.98:0/402166732 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500012430 con 0x7f0504159510 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 -- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0504159510 msgr2=0x7f05040cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0504159510 0x7f05040cb5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0500009e60 tx=0x7f0500009780 comp rx=0 tx=0).stop 2023-10-06T01:40:07.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 -- 172.21.15.98:0/402166732 shutdown_connections 2023-10-06T01:40:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0504159510 0x7f05040cb5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 --2- 172.21.15.98:0/402166732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05041598e0 0x7f05040cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 -- 172.21.15.98:0/402166732 >> 172.21.15.98:0/402166732 conn(0x7f0504089e90 msgr2=0x7f05040997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 -- 172.21.15.98:0/402166732 shutdown_connections 2023-10-06T01:40:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.516+0000 7f050b7fc700 1 -- 172.21.15.98:0/402166732 wait complete. 2023-10-06T01:40:07.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.517+0000 7f050b7fc700 1 Processor -- start 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.517+0000 7f050b7fc700 1 -- start start 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0504159510 0x7f05040c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05041598e0 0x7f05040c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05040ca720 con 0x7f05040ce820 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05040b50d0 con 0x7f05041598e0 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05040b53b0 con 0x7f0504159510 2023-10-06T01:40:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42100/0 (socket says 172.21.15.98:42100) 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 -- 172.21.15.98:0/2858208332 learned_addr learned my addr 172.21.15.98:0/2858208332 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0508d97700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0504159510 0x7f05040c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509d99700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05041598e0 0x7f05040c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 -- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0504159510 msgr2=0x7f05040c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0504159510 0x7f05040c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 -- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05041598e0 msgr2=0x7f05040c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05041598e0 0x7f05040c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 -- 172.21.15.98:0/2858208332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40094f0 con 0x7f05040ce820 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f0509598700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0504067c60 tx=0x7f0500019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500012c80 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0500012e00 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0500009a20 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.518+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000276c0 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.519+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05040b5970 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.519+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0504156210 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.520+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0500024070 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.520+0000 7f04f97fa700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.520+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0500086e40 con 0x7f05040ce820 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.520+0000 7f0508d97700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.520+0000 7f0508d97700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 0x7f04e80693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f050414ec10 tx=0x7f04f4009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.524+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0500053c60 con 0x7f05040ce820 2023-10-06T01:40:07.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.704+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0504064ad0 con 0x7f05040ce820 2023-10-06T01:40:07.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.705+0000 7f04f97fa700 1 -- 172.21.15.98:0/2858208332 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0500057280 con 0x7f05040ce820 2023-10-06T01:40:07.705 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:07.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 msgr2=0x7f04e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 0x7f04e80693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f050414ec10 tx=0x7f04f4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 msgr2=0x7f05040c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0504067c60 tx=0x7f0500019000 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 shutdown_connections 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05041598e0 0x7f05040c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05040ce820 0x7f05040c7600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 --2- 172.21.15.98:0/2858208332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0504159510 0x7f05040c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 >> 172.21.15.98:0/2858208332 conn(0x7f0504089e90 msgr2=0x7f050415bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 shutdown_connections 2023-10-06T01:40:07.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.707+0000 7f050b7fc700 1 -- 172.21.15.98:0/2858208332 wait complete. 2023-10-06T01:40:07.719 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:07.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.835+0000 7f2404d6c700 1 Processor -- start 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.835+0000 7f2404d6c700 1 -- start start 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f2404d6c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f24000ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f2404d6c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24000cbcb0 0x7f24000ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f2404d6c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24000cf610 con 0x7f2400159510 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f2404d6c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24000cf8f0 con 0x7f24000cbcb0 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f2404d6c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24000cfbd0 con 0x7f24001598e0 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fe59c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2400159510 0x7f24000cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43154/0 (socket says 172.21.15.98:43154) 2023-10-06T01:40:07.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fe59c700 1 -- 172.21.15.98:0/2912209484 learned_addr learned my addr 172.21.15.98:0/2912209484 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2912209484 >> v2:172.21.15.184:3300/0 conn(0x7f24000cbcb0 0x7f24000ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f24000ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675708612 0 0) 0x7f24000cf610 con 0x7f2400159510 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23ec003570 con 0x7f2400159510 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.836+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259603758 0 0) 0x7f23ec003570 con 0x7f2400159510 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 msgr2=0x7f24000ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f24000ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 >> v2:172.21.15.184:3300/0 conn(0x7f24000cbcb0 msgr2=0x7f24000ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 --2- 172.21.15.98:0/2912209484 >> v2:172.21.15.184:3300/0 conn(0x7f24000cbcb0 0x7f24000ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000cfeb0 con 0x7f2400159510 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f23f0001470 con 0x7f2400159510 2023-10-06T01:40:07.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2912209484 >> v2:172.21.15.184:3300/0 conn(0x7f24000cbcb0 0x7f24000ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 >> v1:172.21.15.98:6789/0 conn(0x7f2400159510 legacy=0x7f24000cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec003a60 0x7f23ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23ec004810 0x7f23ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f23ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24000cf8f0 con 0x7f23ec004810 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24000cfbd0 con 0x7f23ec003a60 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec00ba60 con 0x7f24001598e0 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23ec004810 0x7f23ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fe59c700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec003a60 0x7f23ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f23ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 -- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 msgr2=0x7f23ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f23ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 -- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec003a60 msgr2=0x7f23ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec003a60 0x7f23ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 -- 172.21.15.98:0/2912209484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec003570 con 0x7f23ec004810 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24001598e0 0x7f23ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23ec004810 0x7f23ec004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009c00 tx=0x7f23e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88f3fb70706d095 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800e710 con 0x7f23ec004810 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e800e870 con 0x7f23ec004810 2023-10-06T01:40:07.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.837+0000 7f23fdd9b700 1 -- 172.21.15.98:0/2912209484 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800ec30 con 0x7f23ec004810 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 -- 172.21.15.98:0/2912209484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23ec004810 msgr2=0x7f23ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2912209484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23ec004810 0x7f23ec004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009c00 tx=0x7f23e80091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f23fed9d700 1 -- 172.21.15.98:0/2912209484 reap_dead start 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 -- 172.21.15.98:0/2912209484 shutdown_connections 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 -- 172.21.15.98:0/2912209484 >> 172.21.15.98:0/2912209484 conn(0x7f2400089e90 msgr2=0x7f24000997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 -- 172.21.15.98:0/2912209484 shutdown_connections 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.838+0000 7f2404d6c700 1 -- 172.21.15.98:0/2912209484 wait complete. 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.839+0000 7f2404d6c700 1 Processor -- start 2023-10-06T01:40:07.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.839+0000 7f2404d6c700 1 -- start start 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec004810 0x7f240009ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f240016df50 0x7f240016d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240009c580 con 0x7f240009ca10 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240009bc70 con 0x7f23ec004810 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f2404d6c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240009b430 con 0x7f240016df50 2023-10-06T01:40:07.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42112/0 (socket says 172.21.15.98:42112) 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 -- 172.21.15.98:0/2508291410 learned_addr learned my addr 172.21.15.98:0/2508291410 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fe59c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec004810 0x7f240009ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23f5d9b700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f240016df50 0x7f240016d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 -- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f240016df50 msgr2=0x7f240016d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f240016df50 0x7f240016d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 -- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec004810 msgr2=0x7f240009ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec004810 0x7f240009ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 -- 172.21.15.98:0/2508291410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240009bf30 con 0x7f240009ca10 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.840+0000 7f23fed9d700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2400088cf0 tx=0x7f23f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800e070 con 0x7f240009ca10 2023-10-06T01:40:07.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8011c50 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8016470 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240009c210 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240016db30 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.841+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240016dd20 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.842+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23f8016860 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.842+0000 7f23f6ffd700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 0x7f23e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.842+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f23f80855a0 con 0x7f240009ca10 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.842+0000 7f23fe59c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 0x7f23e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.843+0000 7f23fe59c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 0x7f23e00693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23f0001d20 tx=0x7f23f0001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:07.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:07.846+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23f8055e50 con 0x7f240009ca10 2023-10-06T01:40:08.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.028+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2400064ad0 con 0x7f240009ca10 2023-10-06T01:40:08.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.029+0000 7f23f6ffd700 1 -- 172.21.15.98:0/2508291410 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23f80559e0 con 0x7f240009ca10 2023-10-06T01:40:08.030 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 msgr2=0x7f23e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 0x7f23e00693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23f0001d20 tx=0x7f23f0001e90 comp rx=0 tx=0).stop 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 msgr2=0x7f24000b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2400088cf0 tx=0x7f23f800b040 comp rx=0 tx=0).stop 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 shutdown_connections 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f23e0066f30 0x7f23e00693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240009ca10 0x7f24000b3360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23ec004810 0x7f240009ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 --2- 172.21.15.98:0/2508291410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f240016df50 0x7f240016d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 >> 172.21.15.98:0/2508291410 conn(0x7f2400089e90 msgr2=0x7f24000c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 shutdown_connections 2023-10-06T01:40:08.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.031+0000 7f2404d6c700 1 -- 172.21.15.98:0/2508291410 wait complete. 2023-10-06T01:40:08.043 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:08.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.162+0000 7f89a177c700 1 Processor -- start 2023-10-06T01:40:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.162+0000 7f89a177c700 1 -- start start 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a177c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f899c0d0a00 0x7f899c164330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a177c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c0cdc30 con 0x7f899c0ccab0 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a177c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c0cb500 con 0x7f899c1539d0 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a177c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0cd020 con 0x7f899c0d0a00 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899b7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f899c0ccab0 0x7f899c0d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43158/0 (socket says 172.21.15.98:43158) 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899b7fe700 1 -- 172.21.15.98:0/3847999202 learned_addr learned my addr 172.21.15.98:0/3847999202 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a0f7b700 1 --2- 172.21.15.98:0/3847999202 >> v2:172.21.15.98:3300/0 conn(0x7f899c0d0a00 0x7f899c164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899affd700 1 -- 172.21.15.98:0/3847999202 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436039884 0 0) 0x7f899c0cdc30 con 0x7f899c0ccab0 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899affd700 1 -- 172.21.15.98:0/3847999202 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8984003570 con 0x7f899c0ccab0 2023-10-06T01:40:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a0f7b700 1 -- 172.21.15.98:0/3847999202 >> v1:172.21.15.184:6789/0 conn(0x7f899c1539d0 legacy=0x7f899c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a0f7b700 1 -- 172.21.15.98:0/3847999202 >> v1:172.21.15.98:6789/0 conn(0x7f899c0ccab0 legacy=0x7f899c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a0f7b700 1 -- 172.21.15.98:0/3847999202 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c0cd300 con 0x7f899c0d0a00 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f89a0f7b700 1 --2- 172.21.15.98:0/3847999202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0d0a00 0x7f899c164330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8994009e60 tx=0x7f8994009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb58f8c04f058892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899affd700 1 -- 172.21.15.98:0/3847999202 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400e070 con 0x7f899c0d0a00 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899affd700 1 -- 172.21.15.98:0/3847999202 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400bb60 con 0x7f899c0d0a00 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.163+0000 7f899affd700 1 -- 172.21.15.98:0/3847999202 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994012430 con 0x7f899c0d0a00 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 -- 172.21.15.98:0/3847999202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0d0a00 msgr2=0x7f899c164330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 --2- 172.21.15.98:0/3847999202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0d0a00 0x7f899c164330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8994009e60 tx=0x7f8994009780 comp rx=0 tx=0).stop 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 -- 172.21.15.98:0/3847999202 shutdown_connections 2023-10-06T01:40:08.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 --2- 172.21.15.98:0/3847999202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0d0a00 0x7f899c164330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 -- 172.21.15.98:0/3847999202 >> 172.21.15.98:0/3847999202 conn(0x7f899c068100 msgr2=0x7f899c0684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 -- 172.21.15.98:0/3847999202 shutdown_connections 2023-10-06T01:40:08.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.164+0000 7f89a177c700 1 -- 172.21.15.98:0/3847999202 wait complete. 2023-10-06T01:40:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.165+0000 7f89a177c700 1 Processor -- start 2023-10-06T01:40:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.165+0000 7f89a177c700 1 -- start start 2023-10-06T01:40:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 0x7f899c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f899c1539d0 0x7f899c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c149f00 con 0x7f899c0ccab0 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0b11f0 con 0x7f899c1539d0 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a177c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0b14d0 con 0x7f899c0d0a00 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42130/0 (socket says 172.21.15.98:42130) 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 -- 172.21.15.98:0/1188271639 learned_addr learned my addr 172.21.15.98:0/1188271639 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f89a0f7b700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f899c1539d0 0x7f899c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899b7fe700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 0x7f899c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 -- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 msgr2=0x7f899c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 0x7f899c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 -- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f899c1539d0 msgr2=0x7f899c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f899c1539d0 0x7f899c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 -- 172.21.15.98:0/1188271639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8994009a20 con 0x7f899c0ccab0 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899b7fe700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 0x7f899c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.166+0000 7f899bfff700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c590 tx=0x7f898c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.167+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c011590 con 0x7f899c0ccab0 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.167+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c011710 con 0x7f899c0ccab0 2023-10-06T01:40:08.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.167+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c0b17b0 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.167+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c011ad0 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.167+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c14a2c0 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.168+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c05f270 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.168+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f898c01b3b0 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.168+0000 7f8983fff700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.168+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f898c083ec0 con 0x7f899c0ccab0 2023-10-06T01:40:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.168+0000 7f899b7fe700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.169+0000 7f899b7fe700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 0x7f89880693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89900058d0 tx=0x7f8990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.172+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f898c050ce0 con 0x7f899c0ccab0 2023-10-06T01:40:08.357 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.356+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f899c15cd90 con 0x7f899c0ccab0 2023-10-06T01:40:08.357 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.357+0000 7f8983fff700 1 -- 172.21.15.98:0/1188271639 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f898c017070 con 0x7f899c0ccab0 2023-10-06T01:40:08.357 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 msgr2=0x7f89880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 0x7f89880693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89900058d0 tx=0x7f8990009040 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 msgr2=0x7f899c1499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c590 tx=0x7f898c00c8a0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 shutdown_connections 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f899c1539d0 0x7f899c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f899c0ccab0 0x7f899c1499c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 --2- 172.21.15.98:0/1188271639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f899c0d0a00 0x7f899c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 >> 172.21.15.98:0/1188271639 conn(0x7f899c068100 msgr2=0x7f899c0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 shutdown_connections 2023-10-06T01:40:08.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.359+0000 7f89a177c700 1 -- 172.21.15.98:0/1188271639 wait complete. 2023-10-06T01:40:08.371 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:08.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.487+0000 7f3ffb59e700 1 Processor -- start 2023-10-06T01:40:08.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 -- start start 2023-10-06T01:40:08.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3ffc0b2660 0x7f3ffc0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc0cc9b0 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc0cbda0 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.488+0000 7f3ffb59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc0cc080 con 0x7f3ffc0b2660 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42144/0 (socket says 172.21.15.98:42144) 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 -- 172.21.15.98:0/729981059 learned_addr learned my addr 172.21.15.98:0/729981059 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ffa59c700 1 --2- 172.21.15.98:0/729981059 >> v2:172.21.15.184:3300/0 conn(0x7f3ffc0b2660 0x7f3ffc0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 -- 172.21.15.98:0/729981059 >> v2:172.21.15.184:3300/0 conn(0x7f3ffc0b2660 msgr2=0x7f3ffc0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 --2- 172.21.15.98:0/729981059 >> v2:172.21.15.184:3300/0 conn(0x7f3ffc0b2660 0x7f3ffc0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 -- 172.21.15.98:0/729981059 >> v1:172.21.15.184:6789/0 conn(0x7f3ffc0ceae0 legacy=0x7f3ffc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:08.491 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 -- 172.21.15.98:0/729981059 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0cc360 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff9d9b700 1 --2- 172.21.15.98:0/729981059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc089500 tx=0x7f3ff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904d0edf24e4c905 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff959a700 1 -- 172.21.15.98:0/729981059 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000b820 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff959a700 1 -- 172.21.15.98:0/729981059 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000b980 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.489+0000 7f3ff959a700 1 -- 172.21.15.98:0/729981059 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000bd20 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 -- 172.21.15.98:0/729981059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0b2a30 msgr2=0x7f3ffc0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/729981059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc089500 tx=0x7f3ff0009250 comp rx=0 tx=0).stop 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 -- 172.21.15.98:0/729981059 shutdown_connections 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/729981059 >> v2:172.21.15.184:3300/0 conn(0x7f3ffc0b2660 0x7f3ffc0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/729981059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0b2a30 0x7f3ffc0cd230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 -- 172.21.15.98:0/729981059 >> 172.21.15.98:0/729981059 conn(0x7f3ffc099560 msgr2=0x7f3ffc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 -- 172.21.15.98:0/729981059 shutdown_connections 2023-10-06T01:40:08.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.490+0000 7f3ffb59e700 1 -- 172.21.15.98:0/729981059 wait complete. 2023-10-06T01:40:08.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.491+0000 7f3ffb59e700 1 Processor -- start 2023-10-06T01:40:08.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.491+0000 7f3ffb59e700 1 -- start start 2023-10-06T01:40:08.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 0x7f3ffc0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ffc0b2a30 0x7f3ffc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc169b50 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0ca680 con 0x7f3ffc0b2660 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc169dc0 con 0x7f3ffc0b2a30 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42146/0 (socket says 172.21.15.98:42146) 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 -- 172.21.15.98:0/3229686453 learned_addr learned my addr 172.21.15.98:0/3229686453 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffa59c700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 0x7f3ffc0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ff9d9b700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ffc0b2a30 0x7f3ffc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 -- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ffc0b2a30 msgr2=0x7f3ffc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ffc0b2a30 0x7f3ffc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 -- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 msgr2=0x7f3ffc0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 0x7f3ffc0c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 -- 172.21.15.98:0/3229686453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00094f0 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffa59c700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 0x7f3ffc0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffad9d700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c930 tx=0x7f3ff40026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400ce70 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff400a780 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc0c9de0 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400ab40 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.492+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc0ca3a0 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.493+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc05f270 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.494+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ff4012420 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.494+0000 7f3fea7fc700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.494+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3ff4084d10 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.494+0000 7f3ffa59c700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.495+0000 7f3ffa59c700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 0x7f3fdc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0b6330 tx=0x7f3fec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.498+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ff4051b30 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.678+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ffc0cc640 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.679+0000 7f3fea7fc700 1 -- 172.21.15.98:0/3229686453 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ff4055150 con 0x7f3ffc0ceae0 2023-10-06T01:40:08.680 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 msgr2=0x7f3fdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 0x7f3fdc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0b6330 tx=0x7f3fec008040 comp rx=0 tx=0).stop 2023-10-06T01:40:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 msgr2=0x7f3ffc154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c930 tx=0x7f3ff40026e0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 shutdown_connections 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ffc0ceae0 0x7f3ffc154590 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ffc0b2660 0x7f3ffc0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 --2- 172.21.15.98:0/3229686453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ffc0b2a30 0x7f3ffc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 >> 172.21.15.98:0/3229686453 conn(0x7f3ffc099560 msgr2=0x7f3ffc0c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 shutdown_connections 2023-10-06T01:40:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.681+0000 7f3ffb59e700 1 -- 172.21.15.98:0/3229686453 wait complete. 2023-10-06T01:40:08.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:08.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.807+0000 7fcf6fe1e700 1 Processor -- start 2023-10-06T01:40:08.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.807+0000 7fcf6fe1e700 1 -- start start 2023-10-06T01:40:08.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf680ca3b0 0x7fcf681582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf680adba0 0x7fcf680ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680ce710 0x7fcf6815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680cbbd0 con 0x7fcf680ca3b0 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf68157e20 con 0x7fcf680adba0 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6fe1e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680ceb50 con 0x7fcf680ce710 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6dbba700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf680ca3b0 0x7fcf681582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6dbba700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf680ca3b0 0x7fcf681582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42148/0 (socket says 172.21.15.98:42148) 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6dbba700 1 -- 172.21.15.98:0/1144764001 learned_addr learned my addr 172.21.15.98:0/1144764001 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6d3b9700 1 --2- 172.21.15.98:0/1144764001 >> v2:172.21.15.184:3300/0 conn(0x7fcf680adba0 0x7fcf680ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.808+0000 7fcf6e3bb700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680ce710 0x7fcf6815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 -- 172.21.15.98:0/1144764001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680ce710 msgr2=0x7fcf6815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680ce710 0x7fcf6815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 -- 172.21.15.98:0/1144764001 >> v2:172.21.15.184:3300/0 conn(0x7fcf680adba0 msgr2=0x7fcf680ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1144764001 >> v2:172.21.15.184:3300/0 conn(0x7fcf680adba0 0x7fcf680ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 -- 172.21.15.98:0/1144764001 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680cb610 con 0x7fcf680ca3b0 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf681582b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806c390 tx=0x7fcf64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a61b1f59e7b905 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6cbb8700 1 -- 172.21.15.98:0/1144764001 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400e070 con 0x7fcf680ca3b0 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6cbb8700 1 -- 172.21.15.98:0/1144764001 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400b970 con 0x7fcf680ca3b0 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6cbb8700 1 -- 172.21.15.98:0/1144764001 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640092e0 con 0x7fcf680ca3b0 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1144764001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 msgr2=0x7fcf681582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf681582b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806c390 tx=0x7fcf64009840 comp rx=0 tx=0).stop 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1144764001 shutdown_connections 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680ce710 0x7fcf6815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1144764001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf681582b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.809+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1144764001 >> v2:172.21.15.184:3300/0 conn(0x7fcf680adba0 0x7fcf680ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.810+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1144764001 >> 172.21.15.98:0/1144764001 conn(0x7fcf68070810 msgr2=0x7fcf68070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:08.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.810+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1144764001 shutdown_connections 2023-10-06T01:40:08.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.810+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1144764001 wait complete. 2023-10-06T01:40:08.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 Processor -- start 2023-10-06T01:40:08.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 -- start start 2023-10-06T01:40:08.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680adba0 0x7fcf68148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf680ce710 0x7fcf680acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c2510 con 0x7fcf680ca3b0 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680694a0 con 0x7fcf680ce710 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6fe1e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c2230 con 0x7fcf680adba0 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.811+0000 7fcf6d3b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42156/0 (socket says 172.21.15.98:42156) 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 -- 172.21.15.98:0/1124514970 learned_addr learned my addr 172.21.15.98:0/1124514970 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680adba0 0x7fcf68148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6e3bb700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf680ce710 0x7fcf680acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 -- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680adba0 msgr2=0x7fcf68148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680adba0 0x7fcf68148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 -- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf680ce710 msgr2=0x7fcf680acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf680ce710 0x7fcf680acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 -- 172.21.15.98:0/1124514970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64009ae0 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6d3b9700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e900 tx=0x7fcf580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800eea0 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800c850 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf68068e50 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800cc10 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.812+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf68153280 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.813+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf68069610 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.814+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf58013470 con 0x7fcf680ca3b0 2023-10-06T01:40:08.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.814+0000 7fcf5dffb700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:08.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.814+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcf58084af0 con 0x7fcf680ca3b0 2023-10-06T01:40:08.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.814+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:08.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.815+0000 7fcf6dbba700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009880 tx=0x7fcf64018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:08.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:08.818+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf58051910 con 0x7fcf680ca3b0 2023-10-06T01:40:09.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.042+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf68064ad0 con 0x7fcf680ca3b0 2023-10-06T01:40:09.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.042+0000 7fcf5dffb700 1 -- 172.21.15.98:0/1124514970 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fcf58054f30 con 0x7fcf680ca3b0 2023-10-06T01:40:09.043 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:09.044 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:09.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 msgr2=0x7fcf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009880 tx=0x7fcf64018000 comp rx=0 tx=0).stop 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 msgr2=0x7fcf680c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e900 tx=0x7fcf580091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 shutdown_connections 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf680ce710 0x7fcf680acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680adba0 0x7fcf68148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 --2- 172.21.15.98:0/1124514970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf680ca3b0 0x7fcf680c4c60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.044+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 >> 172.21.15.98:0/1124514970 conn(0x7fcf68070810 msgr2=0x7fcf680ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:09.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.045+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 shutdown_connections 2023-10-06T01:40:09.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.045+0000 7fcf6fe1e700 1 -- 172.21.15.98:0/1124514970 wait complete. 2023-10-06T01:40:09.047 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:09.058 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:09.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.172+0000 7efd3ccf7700 1 Processor -- start 2023-10-06T01:40:09.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.172+0000 7efd3ccf7700 1 -- start start 2023-10-06T01:40:09.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd3ccf7700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efd380cbc70 0x7efd380cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd3ccf7700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd380b2140 0x7efd380b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd3ccf7700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380cb7e0 con 0x7efd38159480 2023-10-06T01:40:09.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd3ccf7700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380b08a0 con 0x7efd380cbc70 2023-10-06T01:40:09.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd3ccf7700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380cb190 con 0x7efd380b2140 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd377fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efd38159480 0x7efd380b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43172/0 (socket says 172.21.15.98:43172) 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd377fe700 1 -- 172.21.15.98:0/3200793055 learned_addr learned my addr 172.21.15.98:0/3200793055 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd36ffd700 1 --2- 172.21.15.98:0/3200793055 >> v2:172.21.15.98:3300/0 conn(0x7efd380cbc70 0x7efd380cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd37fff700 1 --2- 172.21.15.98:0/3200793055 >> v2:172.21.15.184:3300/0 conn(0x7efd380b2140 0x7efd380b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd367fc700 1 -- 172.21.15.98:0/3200793055 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373363440 0 0) 0x7efd380cb7e0 con 0x7efd38159480 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd367fc700 1 -- 172.21.15.98:0/3200793055 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd20003570 con 0x7efd38159480 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd36ffd700 1 -- 172.21.15.98:0/3200793055 >> v2:172.21.15.184:3300/0 conn(0x7efd380b2140 msgr2=0x7efd380b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd36ffd700 1 --2- 172.21.15.98:0/3200793055 >> v2:172.21.15.184:3300/0 conn(0x7efd380b2140 0x7efd380b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd36ffd700 1 -- 172.21.15.98:0/3200793055 >> v1:172.21.15.98:6789/0 conn(0x7efd38159480 legacy=0x7efd380b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.173+0000 7efd36ffd700 1 -- 172.21.15.98:0/3200793055 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380cb470 con 0x7efd380cbc70 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd36ffd700 1 --2- 172.21.15.98:0/3200793055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd380cbc70 0x7efd380cc0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd380ca830 tx=0x7efd2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e07a7f3393b0e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd367fc700 1 -- 172.21.15.98:0/3200793055 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c011630 con 0x7efd380cbc70 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd367fc700 1 -- 172.21.15.98:0/3200793055 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c011790 con 0x7efd380cbc70 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd367fc700 1 -- 172.21.15.98:0/3200793055 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c011b30 con 0x7efd380cbc70 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 -- 172.21.15.98:0/3200793055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd380cbc70 msgr2=0x7efd380cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/3200793055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd380cbc70 0x7efd380cc0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd380ca830 tx=0x7efd2c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 -- 172.21.15.98:0/3200793055 shutdown_connections 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/3200793055 >> v2:172.21.15.184:3300/0 conn(0x7efd380b2140 0x7efd380b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/3200793055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd380cbc70 0x7efd380cc0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 -- 172.21.15.98:0/3200793055 >> 172.21.15.98:0/3200793055 conn(0x7efd3806ed20 msgr2=0x7efd3806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 -- 172.21.15.98:0/3200793055 shutdown_connections 2023-10-06T01:40:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.174+0000 7efd3ccf7700 1 -- 172.21.15.98:0/3200793055 wait complete. 2023-10-06T01:40:09.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.175+0000 7efd3ccf7700 1 Processor -- start 2023-10-06T01:40:09.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 -- start start 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 0x7efd380d0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd380cbc70 0x7efd381699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38154520 con 0x7efd38159480 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c9d00 con 0x7efd380cbc70 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd3ccf7700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380ce900 con 0x7efd380b2140 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd37fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd37fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42176/0 (socket says 172.21.15.98:42176) 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd37fff700 1 -- 172.21.15.98:0/1526939803 learned_addr learned my addr 172.21.15.98:0/1526939803 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:09.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd36ffd700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd380cbc70 0x7efd381699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.176+0000 7efd377fe700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 0x7efd380d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 -- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 msgr2=0x7efd380d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 0x7efd380d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 -- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd380cbc70 msgr2=0x7efd381699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd380cbc70 0x7efd381699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 -- 172.21.15.98:0/1526939803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009420 con 0x7efd38159480 2023-10-06T01:40:09.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd377fe700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 0x7efd380d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd37fff700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d890 tx=0x7efd30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300125f0 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30012770 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30012b30 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380cdfd0 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.177+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380ce590 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.178+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380ceab0 con 0x7efd38159480 2023-10-06T01:40:09.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.178+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd3001e460 con 0x7efd38159480 2023-10-06T01:40:09.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.179+0000 7efd1f7fe700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.179+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7efd30084f50 con 0x7efd38159480 2023-10-06T01:40:09.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.179+0000 7efd377fe700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.179+0000 7efd377fe700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 0x7efd240693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd38157540 tx=0x7efd28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.183+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd380ceab0 con 0x7efd38159480 2023-10-06T01:40:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.370+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd380cea70 con 0x7efd38159480 2023-10-06T01:40:09.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.370+0000 7efd1f7fe700 1 -- 172.21.15.98:0/1526939803 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7efd30055800 con 0x7efd38159480 2023-10-06T01:40:09.371 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:09.372 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.372+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 msgr2=0x7efd240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.372+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 0x7efd240693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd38157540 tx=0x7efd28009040 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 msgr2=0x7efd380c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d890 tx=0x7efd30008040 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 shutdown_connections 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efd24066f30 0x7efd240693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd38159480 0x7efd380c8e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd380b2140 0x7efd380d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 --2- 172.21.15.98:0/1526939803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd380cbc70 0x7efd381699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 >> 172.21.15.98:0/1526939803 conn(0x7efd3806ed20 msgr2=0x7efd381586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 shutdown_connections 2023-10-06T01:40:09.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.373+0000 7efd3ccf7700 1 -- 172.21.15.98:0/1526939803 wait complete. 2023-10-06T01:40:09.386 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:09.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.500+0000 7f934d25f700 1 Processor -- start 2023-10-06T01:40:09.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.500+0000 7f934d25f700 1 -- start start 2023-10-06T01:40:09.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934d25f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93480cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934d25f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93480b2140 0x7f93480b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934d25f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480cb7e0 con 0x7f9348159480 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934d25f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480b08a0 con 0x7f93480b2140 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934d25f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480cb190 con 0x7f93480cbc70 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f9347fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9348159480 0x7f93480b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43184/0 (socket says 172.21.15.98:43184) 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f9347fff700 1 -- 172.21.15.98:0/2675255270 learned_addr learned my addr 172.21.15.98:0/2675255270 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f934ca5e700 1 --2- 172.21.15.98:0/2675255270 >> v2:172.21.15.184:3300/0 conn(0x7f93480b2140 0x7f93480b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f93477fe700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93480cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108324348 0 0) 0x7f93480cb7e0 con 0x7f9348159480 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.501+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9330003570 con 0x7f9348159480 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643676821 0 0) 0x7f9330003570 con 0x7f9348159480 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 msgr2=0x7f93480cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:09.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93480cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 >> v2:172.21.15.184:3300/0 conn(0x7f93480b2140 msgr2=0x7f93480b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 --2- 172.21.15.98:0/2675255270 >> v2:172.21.15.184:3300/0 conn(0x7f93480b2140 0x7f93480b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480cb470 con 0x7f9348159480 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f9338001470 con 0x7f9348159480 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 >> v1:172.21.15.98:6789/0 conn(0x7f9348159480 legacy=0x7f93480b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330003a60 0x7f9330003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9330004810 0x7f9330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480b08a0 con 0x7f9330004810 2023-10-06T01:40:09.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480cb190 con 0x7f9330003a60 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933000ba60 con 0x7f93480cbc70 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f934ca5e700 1 --2- 172.21.15.98:0/2675255270 >> v2:172.21.15.184:3300/0 conn(0x7f93480b2140 0x7f93480b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9330004810 0x7f9330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9347fff700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330003a60 0x7f9330003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f934ca5e700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 -- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 msgr2=0x7f93300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 -- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330003a60 msgr2=0x7f9330003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330003a60 0x7f9330003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 -- 172.21.15.98:0/2675255270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330003570 con 0x7f9330004810 2023-10-06T01:40:09.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f934ca5e700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480cbc70 0x7f93300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f93477fe700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9330004810 0x7f9330004c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f933c009c30 tx=0x7f933c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5678e6cedd4c05e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00e710 con 0x7f9330004810 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00e870 con 0x7f9330004810 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.502+0000 7f9346ffd700 1 -- 172.21.15.98:0/2675255270 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00ec30 con 0x7f9330004810 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 -- 172.21.15.98:0/2675255270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9330004810 msgr2=0x7f9330004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 --2- 172.21.15.98:0/2675255270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9330004810 0x7f9330004c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f933c009c30 tx=0x7f933c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934ca5e700 1 -- 172.21.15.98:0/2675255270 reap_dead start 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 -- 172.21.15.98:0/2675255270 shutdown_connections 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 -- 172.21.15.98:0/2675255270 >> 172.21.15.98:0/2675255270 conn(0x7f934806ed20 msgr2=0x7f934806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 -- 172.21.15.98:0/2675255270 shutdown_connections 2023-10-06T01:40:09.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.503+0000 7f934d25f700 1 -- 172.21.15.98:0/2675255270 wait complete. 2023-10-06T01:40:09.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.504+0000 7f934d25f700 1 Processor -- start 2023-10-06T01:40:09.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.504+0000 7f934d25f700 1 -- start start 2023-10-06T01:40:09.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330004810 0x7f93480c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480ce530 0x7f93480c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9348169720 con 0x7f93480c9dd0 2023-10-06T01:40:09.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7b60 con 0x7f9330004810 2023-10-06T01:40:09.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7e40 con 0x7f93480ce530 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42194/0 (socket says 172.21.15.98:42194) 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 -- 172.21.15.98:0/4097894401 learned_addr learned my addr 172.21.15.98:0/4097894401 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f9347fff700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330004810 0x7f93480c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934ca5e700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480ce530 0x7f93480c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 -- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480ce530 msgr2=0x7f93480c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480ce530 0x7f93480c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 -- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330004810 msgr2=0x7f93480c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330004810 0x7f93480c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 -- 172.21.15.98:0/4097894401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480c8120 con 0x7f93480c9dd0 2023-10-06T01:40:09.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f93477fe700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f933c00e4b0 tx=0x7f933c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c023070 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00e9e0 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c006e50 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.505+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93480c8400 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.506+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93480c51c0 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.506+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9348169890 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.507+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f933c013470 con 0x7f93480c9dd0 2023-10-06T01:40:09.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.507+0000 7f932ffff700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 0x7f93340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:09.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.507+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f933c085e80 con 0x7f93480c9dd0 2023-10-06T01:40:09.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.507+0000 7f9347fff700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 0x7f93340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:09.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.508+0000 7f9347fff700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 0x7f93340693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9338001d20 tx=0x7f9338001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:09.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.511+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9348169890 con 0x7f93480c9dd0 2023-10-06T01:40:09.735 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.735+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934805f270 con 0x7f93480c9dd0 2023-10-06T01:40:09.736 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.735+0000 7f932ffff700 1 -- 172.21.15.98:0/4097894401 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f933c017020 con 0x7f93480c9dd0 2023-10-06T01:40:09.736 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:09.736 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:09.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 msgr2=0x7f93340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.738 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 0x7f93340693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9338001d20 tx=0x7f9338001e90 comp rx=0 tx=0).stop 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 msgr2=0x7f93480cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f933c00e4b0 tx=0x7f933c01a000 comp rx=0 tx=0).stop 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 shutdown_connections 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9334066f30 0x7f93340693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480ce530 0x7f93480c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330004810 0x7f93480c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 --2- 172.21.15.98:0/4097894401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93480c9dd0 0x7f93480cdff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:09.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 >> 172.21.15.98:0/4097894401 conn(0x7f934806ed20 msgr2=0x7f93481564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:09.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 shutdown_connections 2023-10-06T01:40:09.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:09.737+0000 7f934d25f700 1 -- 172.21.15.98:0/4097894401 wait complete. 2023-10-06T01:40:09.740 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:10.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:39:44.696941+0000', 'modified': '2023-10-06T01:39:58.009938+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/1024246488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6835', 'nonce': 1024246488}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1024246488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T01:40:10.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T01:40:10.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T01:40:10.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.270+0000 7f98ad1a1700 1 Processor -- start 2023-10-06T01:40:10.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.270+0000 7f98ad1a1700 1 -- start start 2023-10-06T01:40:10.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a8159480 0x7f98a80b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98a80b3610 0x7f98a80cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 0x7f98a80b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a80cb7e0 con 0x7f98a80b3610 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a80b08a0 con 0x7f98a80cc1b0 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ad1a1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80cb190 con 0x7f98a8159480 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98a80b3610 0x7f98a80cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98a80b3610 0x7f98a80cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42206/0 (socket says 172.21.15.98:42206) 2023-10-06T01:40:10.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 -- 172.21.15.98:0/351724537 learned_addr learned my addr 172.21.15.98:0/351724537 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a7fff700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a8159480 0x7f98a80b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/351724537 >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 0x7f98a80b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 -- 172.21.15.98:0/351724537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a8159480 msgr2=0x7f98a80b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a8159480 0x7f98a80b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 -- 172.21.15.98:0/351724537 >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 msgr2=0x7f98a80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 --2- 172.21.15.98:0/351724537 >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 0x7f98a80b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 -- 172.21.15.98:0/351724537 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80cb470 con 0x7f98a80b3610 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/351724537 >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 0x7f98a80b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.271+0000 7f98a77fe700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80b3610 0x7f98a80cbc70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98a815cf30 tx=0x7f9898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a763b5aee8cdca7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98a6ffd700 1 -- 172.21.15.98:0/351724537 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800b860 con 0x7f98a80b3610 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98a6ffd700 1 -- 172.21.15.98:0/351724537 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800b9c0 con 0x7f98a80b3610 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98a6ffd700 1 -- 172.21.15.98:0/351724537 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800bd60 con 0x7f98a80b3610 2023-10-06T01:40:10.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 -- 172.21.15.98:0/351724537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80b3610 msgr2=0x7f98a80cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80b3610 0x7f98a80cbc70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98a815cf30 tx=0x7f9898009250 comp rx=0 tx=0).stop 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 -- 172.21.15.98:0/351724537 shutdown_connections 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/351724537 >> v2:172.21.15.184:3300/0 conn(0x7f98a80cc1b0 0x7f98a80b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a8159480 0x7f98a80b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/351724537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80b3610 0x7f98a80cbc70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 -- 172.21.15.98:0/351724537 >> 172.21.15.98:0/351724537 conn(0x7f98a806ed20 msgr2=0x7f98a806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 -- 172.21.15.98:0/351724537 shutdown_connections 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.272+0000 7f98ad1a1700 1 -- 172.21.15.98:0/351724537 wait complete. 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.273+0000 7f98ad1a1700 1 Processor -- start 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.273+0000 7f98ad1a1700 1 -- start start 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a80b3610 0x7f98a8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a8159480 0x7f98a80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80ca130 con 0x7f98a80cc1b0 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80cea10 con 0x7f98a8159480 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ad1a1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80cecf0 con 0x7f98a80b3610 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42220/0 (socket says 172.21.15.98:42220) 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 -- 172.21.15.98:0/3275576953 learned_addr learned my addr 172.21.15.98:0/3275576953 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a80b3610 0x7f98a8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a8159480 0x7f98a80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 -- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a80b3610 msgr2=0x7f98a8154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a80b3610 0x7f98a8154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 -- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a8159480 msgr2=0x7f98a80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a8159480 0x7f98a80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.274+0000 7f98a77fe700 1 -- 172.21.15.98:0/3275576953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980094f0 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98a8069190 tx=0x7f9898017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f988ffff700 1 -- 172.21.15.98:0/3275576953 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898011dc0 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f988ffff700 1 -- 172.21.15.98:0/3275576953 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898024440 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a80cefd0 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f988ffff700 1 -- 172.21.15.98:0/3275576953 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898024800 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.275+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a80cf590 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.276+0000 7f988ffff700 1 -- 172.21.15.98:0/3275576953 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f989801a420 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.277+0000 7f988ffff700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.277+0000 7f988ffff700 1 -- 172.21.15.98:0/3275576953 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9898086890 con 0x7f98a80cc1b0 2023-10-06T01:40:10.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.277+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.277+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 0x7f98940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f989c006fd0 tx=0x7f989c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.278+0000 7f98b76fbb80 1 Processor -- start 2023-10-06T01:40:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 -- start start 2023-10-06T01:40:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x5568908c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 0x5568908c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689081ca70 con 0x5568908e6e50 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689081a270 con 0x5568908c72c0 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556890816060 con 0x5568908c8a10 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.279+0000 7f98a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42226/0 (socket says 172.21.15.98:42226) 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 -- 172.21.15.98:0/775816176 learned_addr learned my addr 172.21.15.98:0/775816176 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a77fe700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x5568908c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 0x5568908c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 -- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 msgr2=0x5568908c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 0x5568908c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 -- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 msgr2=0x5568908c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x5568908c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 -- 172.21.15.98:0/775816176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568906b0270 con 0x5568908e6e50 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 0x5568908c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f98a7fff700 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f989c00fc70 tx=0x7f989c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bac5a6732167d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f988d7fa700 1 -- 172.21.15.98:0/775816176 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0158e0 con 0x5568908e6e50 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f988d7fa700 1 -- 172.21.15.98:0/775816176 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00c040 con 0x5568908e6e50 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.280+0000 7f988d7fa700 1 -- 172.21.15.98:0/775816176 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c015c50 con 0x5568908e6e50 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- 172.21.15.98:0/775816176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 msgr2=0x5568908c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f989c00fc70 tx=0x7f989c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- 172.21.15.98:0/775816176 shutdown_connections 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c8a10 0x5568908c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x5568908c1cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/775816176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x5568908c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- 172.21.15.98:0/775816176 >> 172.21.15.98:0/775816176 conn(0x5568908c6e80 msgr2=0x5568908c76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- 172.21.15.98:0/775816176 shutdown_connections 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- 172.21.15.98:0/775816176 wait complete. 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.281+0000 7f98b76fbb80 1 -- start start 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 Processor -- start 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x55689090fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x55689090ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568907c1890 con 0x5568908c8a10 2023-10-06T01:40:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568907dcea0 con 0x5568908c72c0 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556890698e30 con 0x5568908e6e50 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98b76fbb80 10 client.0 _ll_register_callbacks cb 0x556890889020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42232/0 (socket says 172.21.15.98:42232) 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.282+0000 7f98a77fe700 1 -- 172.21.15.98:0/3882069362 learned_addr learned my addr 172.21.15.98:0/3882069362 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x55689090ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x55689090fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 -- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 msgr2=0x55689090ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x55689090ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 -- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 msgr2=0x55689090fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x55689090fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 -- 172.21.15.98:0/3882069362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c00f8d0 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x55689090ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x55689090fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a77fe700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9898006e20 tx=0x7f9898003040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a9e2820606c929 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989805a5c0 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f989805e310 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568908ba940 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989802b550 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 10 client.4794 fetch_fsmap learned FSMap version 8 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x556890652fe0 con 0x5568908c8a10 2023-10-06T01:40:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568908ba940 con 0x5568908c8a10 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f98980575d0 con 0x5568908c8a10 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 10 client.4794 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 10 client.4794 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 20 client.4794 populate_metadata read hostname 'smithi098' 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x55689091c150 0x55689091c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 4 client.4794 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 --> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556890897870 con 0x55689091c150 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.283+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x55689091c150 0x55689091c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98867fc700 10 client.4794 ms_handle_connect on v2:172.21.15.98:6835/1024246488 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98a7fff700 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x55689091c150 0x55689091c520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7886d56dd632bfac server_cookie=7fb610869517a745 in_seq=0 out_seq=0 2023-10-06T01:40:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98867fc700 1 -- 172.21.15.98:0/3882069362 <== mds.0 v2:172.21.15.98:6835/1024246488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556890897870 con 0x55689091c150 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98867fc700 10 client.4794 handle_command_reply: tid=0 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 client.4794 shutdown 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 msgr2=0x5568908e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9898006e20 tx=0x7f9898003040 comp rx=0 tx=0).stop 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 shutdown_connections 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x55689091c150 0x55689091c520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x55689090ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98867fc700 0 client.4794 ms_handle_reset on v2:172.21.15.98:6835/1024246488 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c72c0 0x55689090fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/3882069362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c8a10 0x5568908e5900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.284+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 >> 172.21.15.98:0/3882069362 conn(0x5568908c6e80 msgr2=0x5568908c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.285+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 shutdown_connections 2023-10-06T01:40:10.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.285+0000 7f98b76fbb80 1 -- 172.21.15.98:0/3882069362 wait complete. 2023-10-06T01:40:10.298 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.285+0000 7f98b76fbb80 20 client.4794 trim_cache size 0 max 16384 2023-10-06T01:40:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.301+0000 7f98b76fbb80 1 Processor -- start 2023-10-06T01:40:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 -- start start 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x55689091b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x5568908bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689091cec0 con 0x5568908c72c0 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568908a52e0 con 0x5568908c8a10 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55689089e1e0 con 0x5568908e6e50 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.302+0000 7f98a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42246/0 (socket says 172.21.15.98:42246) 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 -- 172.21.15.98:0/1697931817 learned_addr learned my addr 172.21.15.98:0/1697931817 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a77fe700 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x55689091b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 -- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 msgr2=0x5568908bc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x5568908bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x5568908bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 -- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 msgr2=0x55689091b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x55689091b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 -- 172.21.15.98:0/1697931817 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568907e4780 con 0x5568908c72c0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f98a7fff700 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f989c00fb40 tx=0x7f989c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641f7fe58f8f17c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f988dffb700 1 -- 172.21.15.98:0/1697931817 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c015a50 con 0x5568908c72c0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f988dffb700 1 -- 172.21.15.98:0/1697931817 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c02e040 con 0x5568908c72c0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.303+0000 7f988dffb700 1 -- 172.21.15.98:0/1697931817 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c015bf0 con 0x5568908c72c0 2023-10-06T01:40:10.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- 172.21.15.98:0/1697931817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 msgr2=0x55689091b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f989c00fb40 tx=0x7f989c020000 comp rx=0 tx=0).stop 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- 172.21.15.98:0/1697931817 shutdown_connections 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908e6e50 0x5568908bc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908c72c0 0x55689091b2d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/1697931817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x55689091b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- 172.21.15.98:0/1697931817 >> 172.21.15.98:0/1697931817 conn(0x5568908c6e80 msgr2=0x5568908c32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- 172.21.15.98:0/1697931817 shutdown_connections 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- 172.21.15.98:0/1697931817 wait complete. 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 -- start start 2023-10-06T01:40:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.304+0000 7f98b76fbb80 1 Processor -- start 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c72c0 0x55689096b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x556890971840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568908e3b80 con 0x5568908e6e50 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556890971e80 con 0x5568908c8a10 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568908e3cf0 con 0x5568908c72c0 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98b76fbb80 10 client.0 _ll_register_callbacks cb 0x556890889020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42248/0 (socket says 172.21.15.98:42248) 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 -- 172.21.15.98:0/2707357200 learned_addr learned my addr 172.21.15.98:0/2707357200 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98a7fff700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c72c0 0x55689096b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98a77fe700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x556890971840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 -- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c72c0 msgr2=0x55689096b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c72c0 0x55689096b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 -- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 msgr2=0x556890971840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x556890971840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.305+0000 7f98ac9a0700 1 -- 172.21.15.98:0/2707357200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568906b0270 con 0x5568908e6e50 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98ac9a0700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98a000df20 tx=0x7f98a0007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c26d495422a744 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0007e50 con 0x5568908e6e50 2023-10-06T01:40:10.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f98a000ce10 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f989c00c040 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98a0019650 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 10 client.4800 fetch_fsmap learned FSMap version 8 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5568907cb470 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00c040 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f98a0019870 con 0x5568908e6e50 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 10 client.4800 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 10 client.4800 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 20 client.4800 populate_metadata read hostname 'smithi098' 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5568908c2000 0x5568908c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 4 client.4800 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-10-06T01:40:10.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 --> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568907b8e00 con 0x5568908c2000 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98a7fff700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5568908c2000 0x5568908c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98867fc700 10 client.4800 ms_handle_connect on v2:172.21.15.98:6835/1024246488 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.306+0000 7f98a7fff700 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5568908c2000 0x5568908c23d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e3bdd0cbf9e61d server_cookie=74a283e360794a25 in_seq=0 out_seq=0 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98867fc700 1 -- 172.21.15.98:0/2707357200 <== mds.0 v2:172.21.15.98:6835/1024246488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5568907b8e00 con 0x5568908c2000 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98867fc700 10 client.4800 handle_command_reply: tid=0 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 client.4800 shutdown 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 msgr2=0x55689091af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98a000df20 tx=0x7f98a0007840 comp rx=0 tx=0).stop 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 shutdown_connections 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568908e6e50 0x55689091af80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.98:6835/1024246488,v1:172.21.15.98:6837/1024246488] conn(0x5568908c2000 0x5568908c23d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568908c72c0 0x55689096b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 --2- 172.21.15.98:0/2707357200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568908c8a10 0x556890971840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98867fc700 0 client.4800 ms_handle_reset on v2:172.21.15.98:6835/1024246488 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 >> 172.21.15.98:0/2707357200 conn(0x5568908c6e80 msgr2=0x5568908e5900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 shutdown_connections 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 1 -- 172.21.15.98:0/2707357200 wait complete. 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98b76fbb80 20 client.4800 trim_cache size 0 max 16384 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "c6dd341a-820f-4e37-8897-3250639f93a1", 2023-10-06T01:40:10.315 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4296, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 24.998199803999999, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 8, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 24, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 30.915213272999999 2023-10-06T01:40:10.316 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 msgr2=0x7f98940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 0x7f98940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f989c006fd0 tx=0x7f989c008040 comp rx=0 tx=0).stop 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 msgr2=0x7f98a80cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98a8069190 tx=0x7f9898017000 comp rx=0 tx=0).stop 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 shutdown_connections 2023-10-06T01:40:10.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a8159480 0x7f98a80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a80b3610 0x7f98a8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 --2- 172.21.15.98:0/3275576953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a80cc1b0 0x7f98a80cde30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.307+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 >> 172.21.15.98:0/3275576953 conn(0x7f98a806ed20 msgr2=0x7f98a81550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.308+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 shutdown_connections 2023-10-06T01:40:10.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.308+0000 7f98ad1a1700 1 -- 172.21.15.98:0/3275576953 wait complete. 2023-10-06T01:40:10.320 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T01:40:10.332 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T01:40:10.332 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T01:40:10.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.333 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:40:10.347 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-06T01:40:10.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T01:40:10.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.422 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2023-10-06T01:40:10.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.519 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:40:10.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2023-10-06T01:40:10.601 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2023-10-06T01:40:10.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:40:10.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.602 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:40:10.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-06T01:40:10.689 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-06T01:40:10.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:40:10.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.691 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:40:10.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:10.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-06T01:40:10.814 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-06T01:40:10.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:40:10.816 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-06T01:40:10.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:40:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.930+0000 7fcf7bc8d700 1 Processor -- start 2023-10-06T01:40:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.930+0000 7fcf7bc8d700 1 -- start start 2023-10-06T01:40:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7bc8d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf7406e690 0x7fcf7406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7bc8d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7bc8d700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7406f040 con 0x7fcf74156540 2023-10-06T01:40:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7bc8d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7406f470 con 0x7fcf7406e690 2023-10-06T01:40:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7bc8d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740acd60 con 0x7fcf740ad1f0 2023-10-06T01:40:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf79228700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf7406e690 0x7fcf7406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf79a29700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcf74156540 0x7fcf740c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43188/0 (socket says 172.21.15.98:43188) 2023-10-06T01:40:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf79a29700 1 -- 172.21.15.98:0/3094683899 learned_addr learned my addr 172.21.15.98:0/3094683899 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.931+0000 7fcf7a22a700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf79228700 1 -- 172.21.15.98:0/3094683899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 msgr2=0x7fcf740ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf79228700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf79228700 1 -- 172.21.15.98:0/3094683899 >> v1:172.21.15.98:6789/0 conn(0x7fcf74156540 legacy=0x7fcf740c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf79228700 1 -- 172.21.15.98:0/3094683899 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74156910 con 0x7fcf7406e690 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf78a27700 1 -- 172.21.15.98:0/3094683899 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205880182 0 0) 0x7fcf7406f040 con 0x7fcf74156540 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf7a22a700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf79228700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf7406eb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf640098e0 tx=0x7fcf640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8087bd389d3cf99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf78a27700 1 -- 172.21.15.98:0/3094683899 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64009e80 con 0x7fcf7406e690 2023-10-06T01:40:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf78a27700 1 -- 172.21.15.98:0/3094683899 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400b860 con 0x7fcf7406e690 2023-10-06T01:40:10.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf78a27700 1 -- 172.21.15.98:0/3094683899 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400bbe0 con 0x7fcf7406e690 2023-10-06T01:40:10.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/3094683899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 msgr2=0x7fcf7406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.932+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf7406eb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf640098e0 tx=0x7fcf640091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/3094683899 shutdown_connections 2023-10-06T01:40:10.944 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/3094683899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf7406eb00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/3094683899 >> 172.21.15.98:0/3094683899 conn(0x7fcf740680c0 msgr2=0x7fcf740cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:10.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/3094683899 shutdown_connections 2023-10-06T01:40:10.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.933+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/3094683899 wait complete. 2023-10-06T01:40:10.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 Processor -- start 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 -- start start 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 0x7fcf74168200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740bfc90 con 0x7fcf7406e690 2023-10-06T01:40:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.934+0000 7fcf7bc8d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740bff70 con 0x7fcf74156540 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf7bc8d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c0250 con 0x7fcf740ad1f0 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42278/0 (socket says 172.21.15.98:42278) 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 -- 172.21.15.98:0/1231056551 learned_addr learned my addr 172.21.15.98:0/1231056551 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf7a22a700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 0x7fcf74168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79228700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 -- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 msgr2=0x7fcf740c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740c0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 -- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 msgr2=0x7fcf74168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:10.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 0x7fcf74168200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:10.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 -- 172.21.15.98:0/1231056551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64009420 con 0x7fcf7406e690 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf7a22a700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 0x7fcf74168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf79a29700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000cd80 tx=0x7fcf70001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000ab10 con 0x7fcf7406e690 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7000ac70 con 0x7fcf7406e690 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740c0530 con 0x7fcf7406e690 2023-10-06T01:40:10.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700070f0 con 0x7fcf7406e690 2023-10-06T01:40:10.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.935+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740c59e0 con 0x7fcf7406e690 2023-10-06T01:40:10.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.936+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf74064ad0 con 0x7fcf7406e690 2023-10-06T01:40:10.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.937+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf700074e0 con 0x7fcf7406e690 2023-10-06T01:40:10.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.937+0000 7fcf69ffb700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 0x7fcf580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:10.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.937+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcf70084eb0 con 0x7fcf7406e690 2023-10-06T01:40:10.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.937+0000 7fcf79228700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 0x7fcf580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:10.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.937+0000 7fcf79228700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 0x7fcf580693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf64003080 tx=0x7fcf64017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:10.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:10.941+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf70051d50 con 0x7fcf7406e690 2023-10-06T01:40:11.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.166+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf740c7b70 con 0x7fcf7406e690 2023-10-06T01:40:11.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.167+0000 7fcf69ffb700 1 -- 172.21.15.98:0/1231056551 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fcf70055370 con 0x7fcf7406e690 2023-10-06T01:40:11.168 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 msgr2=0x7fcf580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 0x7fcf580693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf64003080 tx=0x7fcf64017000 comp rx=0 tx=0).stop 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 msgr2=0x7fcf740c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000cd80 tx=0x7fcf70001ae0 comp rx=0 tx=0).stop 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 shutdown_connections 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcf58066f30 0x7fcf580693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf74156540 0x7fcf74168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf7406e690 0x7fcf740c3b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:11.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 --2- 172.21.15.98:0/1231056551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf740ad1f0 0x7fcf740c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:11.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 >> 172.21.15.98:0/1231056551 conn(0x7fcf740680c0 msgr2=0x7fcf740b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:11.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 shutdown_connections 2023-10-06T01:40:11.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:11.169+0000 7fcf7bc8d700 1 -- 172.21.15.98:0/1231056551 wait complete. 2023-10-06T01:40:11.181 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T01:40:11.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:40:11.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:40:11.182 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:40:11.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.183 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2023-10-06T01:40:11.197 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2023-10-06T01:40:11.198 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41927sec preferred_lft 41927sec 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2023-10-06T01:40:11.199 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:11.200 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T01:40:11.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T01:40:11.200 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:11.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.307 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:40:11.331 INFO:teuthology.orchestra.run.smithi098.stdout:1 2023-10-06T01:40:11.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.332 DEBUG:teuthology.orchestra.run.smithi098:> route 2023-10-06T01:40:11.388 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2023-10-06T01:40:11.388 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:40:11.388 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:40:11.389 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:40:11.389 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:40:11.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.389 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:11.389 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:11.389 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:40:11.389 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:40:11.389 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:40:11.390 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:11.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.488 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:40:11.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.503 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2023-10-06T01:40:11.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.558 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:11.558 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:11.558 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T01:40:11.558 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T01:40:11.558 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:11.647 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T01:40:11.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T01:40:11.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T01:40:11.648 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T01:40:11.648 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:11.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:11.833 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:11.833 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:11.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2023-10-06T01:40:11.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T01:40:11.833 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:11.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:11.926 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:11.940 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:40:11.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:11.941 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:11.994 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:40:11.995 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:40:11.995 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:12.074 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:12.074 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:12.074 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:12.177 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:12.177 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:12.177 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:12.177 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:12.177 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:12.199 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:40:12.200 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:40:12.202 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:40:12.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.202 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:12.267 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-06T01:40:12.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:40:12.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.422+0000 7fc4e7b4f700 1 Processor -- start 2023-10-06T01:40:12.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 -- start start 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4e015caf0 0x7fc4e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4e00caad0 0x7fc4e00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e006ed10 con 0x7fc4e015caf0 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e0067f20 con 0x7fc4e00caad0 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.423+0000 7fc4e7b4f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00cb8b0 con 0x7fc4e00675f0 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4e015caf0 0x7fc4e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.426 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4e015caf0 0x7fc4e006f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46940/0 (socket says 172.21.15.98:46940) 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 -- 172.21.15.98:0/264884515 learned_addr learned my addr 172.21.15.98:0/264884515 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e60ec700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e50ea700 1 --2- 172.21.15.98:0/264884515 >> v2:172.21.15.184:3300/0 conn(0x7fc4e00caad0 0x7fc4e00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 -- 172.21.15.98:0/264884515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 msgr2=0x7fc4e00679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 -- 172.21.15.98:0/264884515 >> v2:172.21.15.184:3300/0 conn(0x7fc4e00caad0 msgr2=0x7fc4e00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/264884515 >> v2:172.21.15.184:3300/0 conn(0x7fc4e00caad0 0x7fc4e00caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 -- 172.21.15.98:0/264884515 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00cbb90 con 0x7fc4e015caf0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e006f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc4e006b940 tx=0x7fc4d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1423d202282450 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e48e9700 1 -- 172.21.15.98:0/264884515 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000e070 con 0x7fc4e015caf0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e48e9700 1 -- 172.21.15.98:0/264884515 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d000ba50 con 0x7fc4e015caf0 2023-10-06T01:40:12.427 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.424+0000 7fc4e48e9700 1 -- 172.21.15.98:0/264884515 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00092e0 con 0x7fc4e015caf0 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/264884515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 msgr2=0x7fc4e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e006f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc4e006b940 tx=0x7fc4d0009840 comp rx=0 tx=0).stop 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/264884515 shutdown_connections 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/264884515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e006f1a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/264884515 >> v2:172.21.15.184:3300/0 conn(0x7fc4e00caad0 0x7fc4e00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/264884515 >> 172.21.15.98:0/264884515 conn(0x7fc4e00cdef0 msgr2=0x7fc4e00c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/264884515 shutdown_connections 2023-10-06T01:40:12.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.425+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/264884515 wait complete. 2023-10-06T01:40:12.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.426+0000 7fc4e7b4f700 1 Processor -- start 2023-10-06T01:40:12.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.426+0000 7fc4e7b4f700 1 -- start start 2023-10-06T01:40:12.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 0x7fc4e0169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00c8b00 con 0x7fc4e015caf0 2023-10-06T01:40:12.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0157ae0 con 0x7fc4e00caad0 2023-10-06T01:40:12.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00b0c90 con 0x7fc4e00675f0 2023-10-06T01:40:12.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46942/0 (socket says 172.21.15.98:46942) 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 -- 172.21.15.98:0/1784711336 learned_addr learned my addr 172.21.15.98:0/1784711336 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e50ea700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 0x7fc4e0169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 -- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 msgr2=0x7fc4e00c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 -- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 msgr2=0x7fc4e0169b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 0x7fc4e0169b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.433 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 -- 172.21.15.98:0/1784711336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0009ae0 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e50ea700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 0x7fc4e0169b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e60ec700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c880 tx=0x7fc4d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8011600 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8011760 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8011ae0 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.427+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e00b0360 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.428+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e00b0920 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.429+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0064ad0 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.429+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4d80193f0 con 0x7fc4e015caf0 2023-10-06T01:40:12.434 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.429+0000 7fc4ddffb700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 0x7fc4cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:12.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.429+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc4d8084ca0 con 0x7fc4e015caf0 2023-10-06T01:40:12.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.429+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 0x7fc4cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:12.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.430+0000 7fc4e58eb700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 0x7fc4cc0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009880 tx=0x7fc4d0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:12.435 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.434+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4d8055550 con 0x7fc4e015caf0 2023-10-06T01:40:12.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.660+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4e006e6c0 con 0x7fc4e015caf0 2023-10-06T01:40:12.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.660+0000 7fc4ddffb700 1 -- 172.21.15.98:0/1784711336 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc4d80550e0 con 0x7fc4e015caf0 2023-10-06T01:40:12.661 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 msgr2=0x7fc4cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 0x7fc4cc0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009880 tx=0x7fc4d0018000 comp rx=0 tx=0).stop 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 msgr2=0x7fc4e01591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c880 tx=0x7fc4d8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 shutdown_connections 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc4cc066f30 0x7fc4cc0693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e015caf0 0x7fc4e01591e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e00675f0 0x7fc4e00c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 --2- 172.21.15.98:0/1784711336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e00caad0 0x7fc4e0169b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 >> 172.21.15.98:0/1784711336 conn(0x7fc4e00cdef0 msgr2=0x7fc4e00c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 shutdown_connections 2023-10-06T01:40:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:12.662+0000 7fc4e7b4f700 1 -- 172.21.15.98:0/1784711336 wait complete. 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:40:12.675 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T01:40:12.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.676 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2023-10-06T01:40:12.690 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:12.691 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41926sec preferred_lft 41926sec 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T01:40:12.692 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c0ec:e9ff:fe71:9101/64 scope link tentative 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::4442:48ff:fe3d:c359/64 scope link tentative 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:12.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.693 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:40:12.753 INFO:teuthology.orchestra.run.smithi098.stdout:1 2023-10-06T01:40:12.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.755 DEBUG:teuthology.orchestra.run.smithi098:> route 2023-10-06T01:40:12.810 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2023-10-06T01:40:12.810 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:40:12.810 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:40:12.810 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:40:12.810 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:40:12.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:40:12.811 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:12.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.911 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:40:12.925 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T01:40:12.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.925 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2023-10-06T01:40:12.980 INFO:teuthology.orchestra.run.smithi098.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T01:40:12.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:12.981 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:12.981 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:12.981 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T01:40:12.981 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T01:40:12.981 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:13.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:13.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T01:40:13.103 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:13.104 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether fe:b4:2f:17:76:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::fcb4:2fff:fe17:76e6/64 scope link 2023-10-06T01:40:13.105 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:13.112 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T01:40:13.192 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:13.192 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:13.192 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:13.192 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether fe:b4:2f:17:76:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.193 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::fcb4:2fff:fe17:76e6/64 scope link 2023-10-06T01:40:13.194 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:13.205 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T01:40:13.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T01:40:13.206 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T01:40:13.207 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T01:40:13.207 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:13.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:13.421 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:13.421 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:13.421 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.1 up 2023-10-06T01:40:13.421 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T01:40:13.421 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:13.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:13.501 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T01:40:13.515 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T01:40:13.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:13.516 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T01:40:13.569 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:40:13.570 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:40:13.570 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T01:40:13.649 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T01:40:13.649 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T01:40:13.649 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:13.657 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T01:40:13.657 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T01:40:13.657 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:13.657 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:13.657 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:13.680 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T01:40:13.680 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T01:40:13.681 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:13.681 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:13.681 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:13.681 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:40:13.682 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:40:13.682 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:40:13.684 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:40:13.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:13.684 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T01:40:13.749 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-06T01:40:13.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:40:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.903+0000 7f9aeb59e700 1 Processor -- start 2023-10-06T01:40:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.903+0000 7f9aeb59e700 1 -- start start 2023-10-06T01:40:13.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aeb59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 0x7f9aec0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aeb59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9aec0c6260 0x7f9aec0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aeb59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec0cc840 con 0x7f9aec1594b0 2023-10-06T01:40:13.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aeb59e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0cbc70 con 0x7f9aec0c6260 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aeb59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0cf440 con 0x7f9aec0cd370 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9aec0c6260 0x7f9aec0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9aec0c6260 0x7f9aec0cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46948/0 (socket says 172.21.15.98:46948) 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 -- 172.21.15.98:0/208103573 learned_addr learned my addr 172.21.15.98:0/208103573 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9ae1d9b700 1 --2- 172.21.15.98:0/208103573 >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 0x7f9aec0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 -- 172.21.15.98:0/208103573 >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 msgr2=0x7f9aec0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 --2- 172.21.15.98:0/208103573 >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 0x7f9aec0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 -- 172.21.15.98:0/208103573 >> v1:172.21.15.184:6789/0 conn(0x7f9aec1594b0 legacy=0x7f9aec0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9aead9d700 1 -- 172.21.15.98:0/208103573 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0cbf10 con 0x7f9aec0c6260 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.904+0000 7f9ae1d9b700 1 --2- 172.21.15.98:0/208103573 >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 0x7f9aec0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:13.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aead9d700 1 --2- 172.21.15.98:0/208103573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec0c6260 0x7f9aec0cef00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9aec06a730 tx=0x7f9ae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d84fe5b769baf10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9ae9d9b700 1 -- 172.21.15.98:0/208103573 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400e070 con 0x7f9aec0c6260 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9ae9d9b700 1 -- 172.21.15.98:0/208103573 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae400ba10 con 0x7f9aec0c6260 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9ae9d9b700 1 -- 172.21.15.98:0/208103573 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40092e0 con 0x7f9aec0c6260 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 -- 172.21.15.98:0/208103573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec0c6260 msgr2=0x7f9aec0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/208103573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec0c6260 0x7f9aec0cef00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9aec06a730 tx=0x7f9ae4009840 comp rx=0 tx=0).stop 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 -- 172.21.15.98:0/208103573 shutdown_connections 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/208103573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec0c6260 0x7f9aec0cef00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/208103573 >> v2:172.21.15.184:3300/0 conn(0x7f9aec0cd370 0x7f9aec0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 -- 172.21.15.98:0/208103573 >> 172.21.15.98:0/208103573 conn(0x7f9aec099600 msgr2=0x7f9aec06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 -- 172.21.15.98:0/208103573 shutdown_connections 2023-10-06T01:40:13.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.905+0000 7f9aeb59e700 1 -- 172.21.15.98:0/208103573 wait complete. 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.906+0000 7f9aeb59e700 1 Processor -- start 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 -- start start 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aec0c6260 0x7f9aec146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aec0cd370 0x7f9aec0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec169270 con 0x7f9aec1594b0 2023-10-06T01:40:13.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec169550 con 0x7f9aec0cd370 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aeb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec169830 con 0x7f9aec0c6260 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aead9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aead9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46964/0 (socket says 172.21.15.98:46964) 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aead9d700 1 -- 172.21.15.98:0/1081759314 learned_addr learned my addr 172.21.15.98:0/1081759314 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9ae1d9b700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aec0cd370 0x7f9aec0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.907+0000 7f9aea59c700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aec0c6260 0x7f9aec146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 -- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aec0c6260 msgr2=0x7f9aec146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aec0c6260 0x7f9aec146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 -- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aec0cd370 msgr2=0x7f9aec0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aec0cd370 0x7f9aec0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 -- 172.21.15.98:0/1081759314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4009ae0 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aead9d700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009fd0 tx=0x7f9ae4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4009840 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae40092e0 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec169b10 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae401d420 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.908+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec147270 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.909+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec05f270 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.909+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ae401d850 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.910+0000 7f9ae2ffd700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 0x7f9acc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.910+0000 7f9aea59c700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 0x7f9acc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.910+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9ae40862b0 con 0x7f9aec1594b0 2023-10-06T01:40:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.910+0000 7f9aea59c700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 0x7f9acc0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9aec15e550 tx=0x7f9adc009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:13.914+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ae40530d0 con 0x7f9aec1594b0 2023-10-06T01:40:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.140+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9aec1693b0 con 0x7f9aec1594b0 2023-10-06T01:40:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.140+0000 7f9ae2ffd700 1 -- 172.21.15.98:0/1081759314 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9aec1693b0 con 0x7f9aec1594b0 2023-10-06T01:40:14.141 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T01:40:14.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 msgr2=0x7f9acc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:14.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 0x7f9acc0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9aec15e550 tx=0x7f9adc009060 comp rx=0 tx=0).stop 2023-10-06T01:40:14.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 msgr2=0x7f9aec0c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:14.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009fd0 tx=0x7f9ae4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:14.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 shutdown_connections 2023-10-06T01:40:14.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9acc066f30 0x7f9acc0693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9aec1594b0 0x7f9aec0c9a80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9aec0c6260 0x7f9aec146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 --2- 172.21.15.98:0/1081759314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aec0cd370 0x7f9aec0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 >> 172.21.15.98:0/1081759314 conn(0x7f9aec099600 msgr2=0x7f9aec0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 shutdown_connections 2023-10-06T01:40:14.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:14.142+0000 7f9aeb59e700 1 -- 172.21.15.98:0/1081759314 wait complete. 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:40:14.155 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T01:40:14.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.156 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-10-06T01:40:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:14.174 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:14.174 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:14.174 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 41919sec preferred_lft 41919sec 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:14.176 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T01:40:14.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.176 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:14.176 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:14.177 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2023-10-06T01:40:14.177 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2023-10-06T01:40:14.177 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2023-10-06T01:40:14.177 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T01:40:14.177 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:14.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.340 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:40:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-06T01:40:14.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.367 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-10-06T01:40:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-10-06T01:40:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:40:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:40:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:40:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:40:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:40:14.437 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:14.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.728 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:40:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.743 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-10-06T01:40:14.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.800 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:14.800 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:14.800 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T01:40:14.800 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T01:40:14.800 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:14.890 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T01:40:14.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T01:40:14.891 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:15.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.076 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:15.076 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:15.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2023-10-06T01:40:15.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T01:40:15.076 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:15.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:15.168 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T01:40:15.182 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T01:40:15.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:15.183 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T01:40:15.237 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:40:15.237 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:40:15.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T01:40:15.336 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T01:40:15.336 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T01:40:15.336 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T01:40:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T01:40:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:15.413 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:15.434 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:40:15.435 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:40:15.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:40:15.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.438 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T01:40:15.461 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-06T01:40:15.461 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:40:15.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.586+0000 7f298d79c700 1 Processor -- start 2023-10-06T01:40:15.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.586+0000 7f298d79c700 1 -- start start 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f298d79c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f29881595a0 0x7f29880aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f298d79c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f29880b07a0 0x7f29880b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f298d79c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29880b0ca0 con 0x7f29880b49e0 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f298d79c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880b0f80 con 0x7f29881595a0 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f298d79c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880b1260 con 0x7f29880b07a0 2023-10-06T01:40:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.587+0000 7f2986ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f29881595a0 0x7f29880aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f297e7fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f29880b49e0 0x7f29880aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56224/0 (socket says 172.21.15.98:56224) 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f297e7fc700 1 -- 172.21.15.98:0/272777028 learned_addr learned my addr 172.21.15.98:0/272777028 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f29877fe700 1 --2- 172.21.15.98:0/272777028 >> v2:172.21.15.184:3300/0 conn(0x7f29880b07a0 0x7f29880b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f2986ffd700 1 -- 172.21.15.98:0/272777028 >> v2:172.21.15.184:3300/0 conn(0x7f29880b07a0 msgr2=0x7f29880b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f2986ffd700 1 --2- 172.21.15.98:0/272777028 >> v2:172.21.15.184:3300/0 conn(0x7f29880b07a0 0x7f29880b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f2986ffd700 1 -- 172.21.15.98:0/272777028 >> v1:172.21.15.98:6789/0 conn(0x7f29880b49e0 legacy=0x7f29880aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f2986ffd700 1 -- 172.21.15.98:0/272777028 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880b1540 con 0x7f29881595a0 2023-10-06T01:40:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f29867fc700 1 -- 172.21.15.98:0/272777028 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688368352 0 0) 0x7f29880b0ca0 con 0x7f29880b49e0 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f2986ffd700 1 --2- 172.21.15.98:0/272777028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f29880aa570 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2978009eb0 tx=0x7f2978009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae05dee150989f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f29867fc700 1 -- 172.21.15.98:0/272777028 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800e070 con 0x7f29881595a0 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f29867fc700 1 -- 172.21.15.98:0/272777028 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800bbf0 con 0x7f29881595a0 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.588+0000 7f29867fc700 1 -- 172.21.15.98:0/272777028 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978012430 con 0x7f29881595a0 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 -- 172.21.15.98:0/272777028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 msgr2=0x7f29880aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 --2- 172.21.15.98:0/272777028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f29880aa570 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2978009eb0 tx=0x7f2978009780 comp rx=0 tx=0).stop 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 -- 172.21.15.98:0/272777028 shutdown_connections 2023-10-06T01:40:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 --2- 172.21.15.98:0/272777028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f29880aa570 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 --2- 172.21.15.98:0/272777028 >> v2:172.21.15.184:3300/0 conn(0x7f29880b07a0 0x7f29880b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 -- 172.21.15.98:0/272777028 >> 172.21.15.98:0/272777028 conn(0x7f2988099830 msgr2=0x7f298806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 -- 172.21.15.98:0/272777028 shutdown_connections 2023-10-06T01:40:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.589+0000 7f298d79c700 1 -- 172.21.15.98:0/272777028 wait complete. 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.590+0000 7f298d79c700 1 Processor -- start 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.590+0000 7f298d79c700 1 -- start start 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29880b07a0 0x7f298816cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 0x7f298809a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298809b6c0 con 0x7f29881595a0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298815ab70 con 0x7f29880b49e0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298815ae10 con 0x7f29880b07a0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46992/0 (socket says 172.21.15.98:46992) 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 -- 172.21.15.98:0/1707815937 learned_addr learned my addr 172.21.15.98:0/1707815937 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f297e7fc700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 0x7f298809a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f2986ffd700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29880b07a0 0x7f298816cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 -- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29880b07a0 msgr2=0x7f298816cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29880b07a0 0x7f298816cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 -- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 msgr2=0x7f298809a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 0x7f298809a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 -- 172.21.15.98:0/1707815937 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978009a20 con 0x7f29881595a0 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f297e7fc700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 0x7f298809a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f29877fe700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f298000ea50 tx=0x7f29800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000ee90 con 0x7f29881595a0 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298000c850 con 0x7f29881595a0 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.591+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298815b0f0 con 0x7f29881595a0 2023-10-06T01:40:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.592+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000cc10 con 0x7f29881595a0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.592+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298815b6b0 con 0x7f29881595a0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.592+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298806a290 con 0x7f29881595a0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.593+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2980013470 con 0x7f29881595a0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.593+0000 7f297f7fe700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 0x7f29680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.593+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2980084b50 con 0x7f29881595a0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.593+0000 7f2986ffd700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 0x7f29680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.594+0000 7f2986ffd700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 0x7f29680693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29780099f0 tx=0x7f2978019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:15.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.597+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2980051970 con 0x7f29881595a0 2023-10-06T01:40:15.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.820+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2988064ad0 con 0x7f29881595a0 2023-10-06T01:40:15.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.820+0000 7f297f7fe700 1 -- 172.21.15.98:0/1707815937 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2980054f90 con 0x7f29881595a0 2023-10-06T01:40:15.821 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T01:40:15.822 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 msgr2=0x7f29680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 0x7f29680693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29780099f0 tx=0x7f2978019000 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 msgr2=0x7f298809ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f298000ea50 tx=0x7f29800091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 shutdown_connections 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2968066f30 0x7f29680693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29881595a0 0x7f298809ae00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29880b07a0 0x7f298816cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 --2- 172.21.15.98:0/1707815937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29880b49e0 0x7f298809a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 >> 172.21.15.98:0/1707815937 conn(0x7f2988099830 msgr2=0x7f29880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 shutdown_connections 2023-10-06T01:40:15.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:15.822+0000 7f298d79c700 1 -- 172.21.15.98:0/1707815937 wait complete. 2023-10-06T01:40:15.834 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T01:40:15.834 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:40:15.834 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:40:15.835 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T01:40:15.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.835 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.851 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 41917sec preferred_lft 41917sec 2023-10-06T01:40:15.852 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 76:c3:57:24:35:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::2c40:c2ff:fe50:5871/64 scope link tentative 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 76:c3:57:24:35:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::74c3:57ff:fe24:3542/64 scope link tentative 2023-10-06T01:40:15.853 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:15.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.854 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:40:15.914 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-06T01:40:15.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.916 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:40:15.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:40:15.973 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:16.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.072 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:40:16.087 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T01:40:16.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.088 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-10-06T01:40:16.142 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T01:40:16.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.143 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:16.143 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:16.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T01:40:16.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T01:40:16.144 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:16.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.231 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:40:16.270 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether ea:44:0b:61:c5:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T01:40:16.271 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T01:40:16.271 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.271 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::e844:bff:fe61:c503/64 scope link tentative 2023-10-06T01:40:16.271 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T01:40:16.368 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:16.368 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:16.368 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:16.368 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.368 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether ea:44:0b:61:c5:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::e844:bff:fe61:c503/64 scope link tentative 2023-10-06T01:40:16.369 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:16.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T01:40:16.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.384 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:16.384 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:16.384 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T01:40:16.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T01:40:16.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T01:40:16.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T01:40:16.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T01:40:16.385 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:16.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.619 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-06T01:40:16.619 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-06T01:40:16.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2023-10-06T01:40:16.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T01:40:16.619 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-06T01:40:16.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:16.698 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T01:40:16.713 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T01:40:16.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:16.713 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T01:40:16.767 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:40:16.767 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:40:16.767 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T01:40:16.847 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T01:40:16.847 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T01:40:16.847 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:16.855 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T01:40:16.855 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T01:40:16.855 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:16.855 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:16.856 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:16.878 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:40:16.879 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:40:16.879 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:40:16.880 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:40:16.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:16.880 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T01:40:16.944 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T01:40:16.960 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:17.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 Processor -- start 2023-10-06T01:40:17.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 -- start start 2023-10-06T01:40:17.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffb101645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb100b0de0 con 0x7ffb100b2620 2023-10-06T01:40:17.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb100cbfe0 con 0x7ffb100b29f0 2023-10-06T01:40:17.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb154ca700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100cb3d0 con 0x7ffb100cd480 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb0effd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ffb100b2620 0x7ffb100b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56234/0 (socket says 172.21.15.98:56234) 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.083+0000 7ffb0effd700 1 -- 172.21.15.98:0/166783869 learned_addr learned my addr 172.21.15.98:0/166783869 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffb101645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596175685 0 0) 0x7ffb00001610 con 0x7ffb100b2620 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf8003570 con 0x7ffb100b2620 2023-10-06T01:40:17.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802067153 0 0) 0x7ffb100cbfe0 con 0x7ffb100b29f0 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb00001610 con 0x7ffb100b29f0 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495875290 0 0) 0x7ffaf8003570 con 0x7ffb100b2620 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 msgr2=0x7ffb101645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffb101645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 >> v1:172.21.15.184:6789/0 conn(0x7ffb100b29f0 legacy=0x7ffb10159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100cb6b0 con 0x7ffb100b2620 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7ffb00001b00 con 0x7ffb100b2620 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffb101645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 >> v1:172.21.15.98:6789/0 conn(0x7ffb100b2620 legacy=0x7ffb100b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.089 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf8003c20 0x7ffaf8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffaf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffaf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100cb3d0 con 0x7ffaf8004d50 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800bb10 con 0x7ffaf8003c20 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800bcd0 con 0x7ffb100cd480 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0effd700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf8003c20 0x7ffaf8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffaf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0e7fc700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffaf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 -- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 msgr2=0x7ffaf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:17.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb100cd480 0x7ffaf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 -- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf8003c20 msgr2=0x7ffaf8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.084+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf8003c20 0x7ffaf8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0f7fe700 1 -- 172.21.15.98:0/166783869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08009420 con 0x7ffaf8004d50 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffaf80051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009df0 tx=0x7ffb080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa03bd4e2ef7b6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08009740 con 0x7ffaf8004d50 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0800e920 con 0x7ffaf8004d50 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0dffb700 1 -- 172.21.15.98:0/166783869 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800ece0 con 0x7ffaf8004d50 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 -- 172.21.15.98:0/166783869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 msgr2=0x7ffaf80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 --2- 172.21.15.98:0/166783869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffaf80051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009df0 tx=0x7ffb080091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb0f7fe700 1 -- 172.21.15.98:0/166783869 reap_dead start 2023-10-06T01:40:17.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 -- 172.21.15.98:0/166783869 shutdown_connections 2023-10-06T01:40:17.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 -- 172.21.15.98:0/166783869 >> 172.21.15.98:0/166783869 conn(0x7ffb1006ed20 msgr2=0x7ffb1006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 -- 172.21.15.98:0/166783869 shutdown_connections 2023-10-06T01:40:17.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.085+0000 7ffb154ca700 1 -- 172.21.15.98:0/166783869 wait complete. 2023-10-06T01:40:17.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 Processor -- start 2023-10-06T01:40:17.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 -- start start 2023-10-06T01:40:17.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb1014ac60 0x7ffb10152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1014b0d0 0x7ffb10153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c8400 con 0x7ffaf8004d50 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c7510 con 0x7ffb1014b0d0 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.087+0000 7ffb154ca700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100c77f0 con 0x7ffb1014ac60 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47004/0 (socket says 172.21.15.98:47004) 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 -- 172.21.15.98:0/3305078874 learned_addr learned my addr 172.21.15.98:0/3305078874 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0f7fe700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1014b0d0 0x7ffb10153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0e7fc700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb1014ac60 0x7ffb10152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 -- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb1014ac60 msgr2=0x7ffb10152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb1014ac60 0x7ffb10152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 -- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1014b0d0 msgr2=0x7ffb10153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1014b0d0 0x7ffb10153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 -- 172.21.15.98:0/3305078874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100c7ad0 con 0x7ffaf8004d50 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb0effd700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb00007aa0 tx=0x7ffb00007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00007ad0 con 0x7ffaf8004d50 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0000fc60 con 0x7ffaf8004d50 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00014470 con 0x7ffaf8004d50 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb100c7db0 con 0x7ffaf8004d50 2023-10-06T01:40:17.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.088+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1016ad30 con 0x7ffaf8004d50 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.089+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb10064ad0 con 0x7ffaf8004d50 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.090+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb000147e0 con 0x7ffaf8004d50 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.090+0000 7ffaf6ffd700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 0x7ffafc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.090+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ffb000852d0 con 0x7ffaf8004d50 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.090+0000 7ffb0e7fc700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 0x7ffafc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.090+0000 7ffb0e7fc700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 0x7ffafc0693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb1006f290 tx=0x7ffb04007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.094+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb000521b0 con 0x7ffaf8004d50 2023-10-06T01:40:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.278+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb100b0de0 con 0x7ffaf8004d50 2023-10-06T01:40:17.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.278+0000 7ffaf6ffd700 1 -- 172.21.15.98:0/3305078874 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7ffb100b0de0 con 0x7ffaf8004d50 2023-10-06T01:40:17.279 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:17.279 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 msgr2=0x7ffafc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 0x7ffafc0693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb1006f290 tx=0x7ffb04007d30 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 msgr2=0x7ffb1016a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb00007aa0 tx=0x7ffb00007db0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 shutdown_connections 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ffafc066f30 0x7ffafc0693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1014b0d0 0x7ffb10153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf8004d50 0x7ffb1016a7f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 --2- 172.21.15.98:0/3305078874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb1014ac60 0x7ffb10152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 >> 172.21.15.98:0/3305078874 conn(0x7ffb1006ed20 msgr2=0x7ffb100b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 shutdown_connections 2023-10-06T01:40:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.281+0000 7ffb154ca700 1 -- 172.21.15.98:0/3305078874 wait complete. 2023-10-06T01:40:17.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:17.294 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:17.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.413+0000 7f04a44a0700 1 Processor -- start 2023-10-06T01:40:17.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.413+0000 7f04a44a0700 1 -- start start 2023-10-06T01:40:17.416 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a44a0700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f049c1598e0 0x7f049c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.416 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a44a0700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 0x7f049c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.416 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a44a0700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049c0cf610 con 0x7f049c159510 2023-10-06T01:40:17.416 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a44a0700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c0cf8f0 con 0x7f049c1598e0 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a44a0700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c0cfbd0 con 0x7f049c0cbcb0 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a1a3b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f049c1598e0 0x7f049c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a223c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f049c159510 0x7f049c0cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56240/0 (socket says 172.21.15.98:56240) 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a223c700 1 -- 172.21.15.98:0/2911018581 learned_addr learned my addr 172.21.15.98:0/2911018581 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.414+0000 7f04a2a3d700 1 --2- 172.21.15.98:0/2911018581 >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 0x7f049c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a1a3b700 1 -- 172.21.15.98:0/2911018581 >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 msgr2=0x7f049c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a1a3b700 1 --2- 172.21.15.98:0/2911018581 >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 0x7f049c0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a1a3b700 1 -- 172.21.15.98:0/2911018581 >> v1:172.21.15.98:6789/0 conn(0x7f049c159510 legacy=0x7f049c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a1a3b700 1 -- 172.21.15.98:0/2911018581 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c0cfeb0 con 0x7f049c1598e0 2023-10-06T01:40:17.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a2a3d700 1 --2- 172.21.15.98:0/2911018581 >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 0x7f049c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a123a700 1 -- 172.21.15.98:0/2911018581 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340192654 0 0) 0x7f049c0cf610 con 0x7f049c159510 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a1a3b700 1 --2- 172.21.15.98:0/2911018581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c1598e0 0x7f049c0ce820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f049c06f990 tx=0x7f0490009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9e59692d89f5ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a123a700 1 -- 172.21.15.98:0/2911018581 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049000e070 con 0x7f049c1598e0 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a123a700 1 -- 172.21.15.98:0/2911018581 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049000bc30 con 0x7f049c1598e0 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a123a700 1 -- 172.21.15.98:0/2911018581 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490012430 con 0x7f049c1598e0 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 -- 172.21.15.98:0/2911018581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c1598e0 msgr2=0x7f049c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 --2- 172.21.15.98:0/2911018581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c1598e0 0x7f049c0ce820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f049c06f990 tx=0x7f0490009780 comp rx=0 tx=0).stop 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 -- 172.21.15.98:0/2911018581 shutdown_connections 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 --2- 172.21.15.98:0/2911018581 >> v2:172.21.15.184:3300/0 conn(0x7f049c0cbcb0 0x7f049c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 --2- 172.21.15.98:0/2911018581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c1598e0 0x7f049c0ce820 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.415+0000 7f04a44a0700 1 -- 172.21.15.98:0/2911018581 >> 172.21.15.98:0/2911018581 conn(0x7f049c089e90 msgr2=0x7f049c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.416+0000 7f04a44a0700 1 -- 172.21.15.98:0/2911018581 shutdown_connections 2023-10-06T01:40:17.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.416+0000 7f04a44a0700 1 -- 172.21.15.98:0/2911018581 wait complete. 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 Processor -- start 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 -- start start 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f049c0cbcb0 0x7f049c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 0x7f049c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c15bd20 con 0x7f049c159510 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c159da0 con 0x7f049c1598e0 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a44a0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c16adc0 con 0x7f049c0cbcb0 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a1a3b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a1a3b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47026/0 (socket says 172.21.15.98:47026) 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.417+0000 7f04a1a3b700 1 -- 172.21.15.98:0/3423998067 learned_addr learned my addr 172.21.15.98:0/3423998067 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a2a3d700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 0x7f049c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 -- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f049c0cbcb0 msgr2=0x7f049c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f049c0cbcb0 0x7f049c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 -- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 msgr2=0x7f049c069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.422 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 0x7f049c069380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 -- 172.21.15.98:0/3423998067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490009a20 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a2a3d700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 0x7f049c069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a1a3b700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0490013000 tx=0x7f0490019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490012e50 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0490027470 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c15a040 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490027830 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.418+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c16b0a0 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.419+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c15a560 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.419+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0490024070 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.420+0000 7f049a7fc700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.420+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0490086f20 con 0x7f049c159510 2023-10-06T01:40:17.423 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.420+0000 7f04a223c700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.424 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.420+0000 7f04a223c700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 0x7f04880693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f04940016f0 tx=0x7f04940079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.424 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.424+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f049c15a560 con 0x7f049c159510 2023-10-06T01:40:17.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.607+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f049c15a560 con 0x7f049c159510 2023-10-06T01:40:17.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.607+0000 7f049a7fc700 1 -- 172.21.15.98:0/3423998067 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f049c15a560 con 0x7f049c159510 2023-10-06T01:40:17.608 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:17.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.609+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 msgr2=0x7f04880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.609+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 0x7f04880693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f04940016f0 tx=0x7f04940079d0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.609+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 msgr2=0x7f049c15a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.609+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0490013000 tx=0x7f0490019000 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 shutdown_connections 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f049c1598e0 0x7f049c069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f049c0cbcb0 0x7f049c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 --2- 172.21.15.98:0/3423998067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f049c159510 0x7f049c15a840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 >> 172.21.15.98:0/3423998067 conn(0x7f049c089e90 msgr2=0x7f049c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 shutdown_connections 2023-10-06T01:40:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.610+0000 7f04a44a0700 1 -- 172.21.15.98:0/3423998067 wait complete. 2023-10-06T01:40:17.622 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:17.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.738+0000 7ff1c223d700 1 Processor -- start 2023-10-06T01:40:17.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.738+0000 7ff1c223d700 1 -- start start 2023-10-06T01:40:17.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1c223d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1c223d700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc0cbf00 con 0x7ff1bc1598e0 2023-10-06T01:40:17.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1c223d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc0cf610 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1c223d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc0cf8f0 con 0x7ff1bc159510 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1baffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff1bc1598e0 0x7ff1bc0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56246/0 (socket says 172.21.15.98:56246) 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1baffd700 1 -- 172.21.15.98:0/692895969 learned_addr learned my addr 172.21.15.98:0/692895969 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1bb7fe700 1 --2- 172.21.15.98:0/692895969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598386592 0 0) 0x7ff1bc0cbf00 con 0x7ff1bc1598e0 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a8003570 con 0x7ff1bc1598e0 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743472791 0 0) 0x7ff1bc0cf610 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1bc0cbf00 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990221061 0 0) 0x7ff1a8003570 con 0x7ff1bc1598e0 2023-10-06T01:40:17.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/692895969 >> v1:172.21.15.184:6789/0 conn(0x7ff1bc0cbae0 legacy=0x7ff1bc0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/692895969 >> v1:172.21.15.98:6789/0 conn(0x7ff1bc1598e0 legacy=0x7ff1bc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/692895969 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0cfbd0 con 0x7ff1bc159510 2023-10-06T01:40:17.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.739+0000 7ff1bb7fe700 1 --2- 172.21.15.98:0/692895969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc0cb5a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009910 tx=0x7ff1a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea41a6e64f65b2e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac001470 con 0x7ff1bc159510 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4011420 con 0x7ff1bc159510 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1ba7fc700 1 -- 172.21.15.98:0/692895969 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40117c0 con 0x7ff1bc159510 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 -- 172.21.15.98:0/692895969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 msgr2=0x7ff1bc0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 --2- 172.21.15.98:0/692895969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc0cb5a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009910 tx=0x7ff1a40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 -- 172.21.15.98:0/692895969 shutdown_connections 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 --2- 172.21.15.98:0/692895969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc0cb5a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 -- 172.21.15.98:0/692895969 >> 172.21.15.98:0/692895969 conn(0x7ff1bc089e90 msgr2=0x7ff1bc0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 -- 172.21.15.98:0/692895969 shutdown_connections 2023-10-06T01:40:17.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.740+0000 7ff1c223d700 1 -- 172.21.15.98:0/692895969 wait complete. 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.741+0000 7ff1c223d700 1 Processor -- start 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.741+0000 7ff1c223d700 1 -- start start 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc09a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1bc1598e0 0x7ff1bc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09d1c0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09d4a0 con 0x7ff1bc1598e0 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1c223d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09d780 con 0x7ff1bc159510 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47042/0 (socket says 172.21.15.98:47042) 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/1331131352 learned_addr learned my addr 172.21.15.98:0/1331131352 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:17.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1baffd700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bbfff700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1bc1598e0 0x7ff1bc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 msgr2=0x7ff1bc09a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc09a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1bc1598e0 msgr2=0x7ff1bc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1bc1598e0 0x7ff1bc14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.742+0000 7ff1bb7fe700 1 -- 172.21.15.98:0/1331131352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4009420 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1bb7fe700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40091e0 tx=0x7ff1a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401cb30 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a401cc90 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc09da60 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401a6a0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.743+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc15c5e0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.744+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc15c7d0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.744+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1a401aa90 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.744+0000 7ff1b37fe700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 0x7ff19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.745+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff1a40867c0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.745+0000 7ff1baffd700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 0x7ff19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.745+0000 7ff1baffd700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 0x7ff19c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc069ec0 tx=0x7ff1ac000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.749+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1a4053540 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.930+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1bc064ad0 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.930+0000 7ff1b37fe700 1 -- 172.21.15.98:0/1331131352 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1a4056b60 con 0x7ff1bc0cbae0 2023-10-06T01:40:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 msgr2=0x7ff19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 0x7ff19c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc069ec0 tx=0x7ff1ac000bc0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 msgr2=0x7ff1bc15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40091e0 tx=0x7ff1a4016000 comp rx=0 tx=0).stop 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 shutdown_connections 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff19c066f30 0x7ff19c0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1bc1598e0 0x7ff1bc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1bc0cbae0 0x7ff1bc15c0a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 --2- 172.21.15.98:0/1331131352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1bc159510 0x7ff1bc09a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:17.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 >> 172.21.15.98:0/1331131352 conn(0x7ff1bc089e90 msgr2=0x7ff1bc0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:17.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 shutdown_connections 2023-10-06T01:40:17.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:17.932+0000 7ff1c223d700 1 -- 172.21.15.98:0/1331131352 wait complete. 2023-10-06T01:40:17.946 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:18.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.063+0000 7f074eebe700 1 Processor -- start 2023-10-06T01:40:18.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.063+0000 7f074eebe700 1 -- start start 2023-10-06T01:40:18.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074eebe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f07480cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074eebe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07480cbae0 0x7f07480cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074eebe700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cbf00 con 0x7f07481598e0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074eebe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480cf610 con 0x7f07480cbae0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074eebe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480cf8f0 con 0x7f0748159510 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f073ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f07481598e0 0x7f07480ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56256/0 (socket says 172.21.15.98:56256) 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f073ffff700 1 -- 172.21.15.98:0/1892178800 learned_addr learned my addr 172.21.15.98:0/1892178800 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891970115 0 0) 0x7f07480cbf00 con 0x7f07481598e0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0730003570 con 0x7f07481598e0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074cc5a700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f07480cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f074d45b700 1 --2- 172.21.15.98:0/1892178800 >> v2:172.21.15.184:3300/0 conn(0x7f07480cbae0 0x7f07480cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.064+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949728557 0 0) 0x7f0730003570 con 0x7f07481598e0 2023-10-06T01:40:18.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 msgr2=0x7f07480cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f07480cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 >> v2:172.21.15.184:3300/0 conn(0x7f07480cbae0 msgr2=0x7f07480cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 --2- 172.21.15.98:0/1892178800 >> v2:172.21.15.184:3300/0 conn(0x7f07480cbae0 0x7f07480cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480cfbd0 con 0x7f07481598e0 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f0738001470 con 0x7f07481598e0 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074cc5a700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f07480cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 >> v1:172.21.15.98:6789/0 conn(0x7f07481598e0 legacy=0x7f07480ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f0730003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0730003fa0 0x7f0730004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0730004d50 0x7f07300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480cf610 con 0x7f0730004d50 2023-10-06T01:40:18.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480cf8f0 con 0x7f0730003fa0 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073000bb10 con 0x7f0748159510 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0730004d50 0x7f07300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074cc5a700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f0730003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073ffff700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0730003fa0 0x7f0730004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 -- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 msgr2=0x7f0730003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0748159510 0x7f0730003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 -- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0730003fa0 msgr2=0x7f0730004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0730003fa0 0x7f0730004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 -- 172.21.15.98:0/1892178800 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0730003570 con 0x7f0730004d50 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f074d45b700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0730004d50 0x7f07300051c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0740006f00 tx=0x7f07400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9c51fb83252d43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074000e710 con 0x7f0730004d50 2023-10-06T01:40:18.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074000e870 con 0x7f0730004d50 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.065+0000 7f073f7fe700 1 -- 172.21.15.98:0/1892178800 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074000ec30 con 0x7f0730004d50 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 -- 172.21.15.98:0/1892178800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0730004d50 msgr2=0x7f07300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 --2- 172.21.15.98:0/1892178800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0730004d50 0x7f07300051c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0740006f00 tx=0x7f07400091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074d45b700 1 -- 172.21.15.98:0/1892178800 reap_dead start 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 -- 172.21.15.98:0/1892178800 shutdown_connections 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 -- 172.21.15.98:0/1892178800 >> 172.21.15.98:0/1892178800 conn(0x7f0748089e90 msgr2=0x7f07480997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 -- 172.21.15.98:0/1892178800 shutdown_connections 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.066+0000 7f074eebe700 1 -- 172.21.15.98:0/1892178800 wait complete. 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.067+0000 7f074eebe700 1 Processor -- start 2023-10-06T01:40:18.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.067+0000 7f074eebe700 1 -- start start 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074814d0a0 0x7f07480b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f074815b4f0 0x7f074814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b4490 con 0x7f074814dba0 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074815b970 con 0x7f074814d0a0 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074eebe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0748169b70 con 0x7f074815b4f0 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47056/0 (socket says 172.21.15.98:47056) 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 -- 172.21.15.98:0/2676058473 learned_addr learned my addr 172.21.15.98:0/2676058473 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074cc5a700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074814d0a0 0x7f07480b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f074d45b700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f074815b4f0 0x7f074814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 -- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f074815b4f0 msgr2=0x7f074814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f074815b4f0 0x7f074814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 -- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074814d0a0 msgr2=0x7f07480b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074814d0a0 0x7f07480b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 -- 172.21.15.98:0/2676058473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0748169de0 con 0x7f074814dba0 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.068+0000 7f073ffff700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07380073f0 tx=0x7f0738001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380077d0 con 0x7f074814dba0 2023-10-06T01:40:18.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073800f870 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07480b3bf0 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073800fc30 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07480b41b0 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.069+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0748064ad0 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.070+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0738014450 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.070+0000 7f073cff9700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 0x7f0734069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.070+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f07380841c0 con 0x7f074814dba0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.070+0000 7f074cc5a700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 0x7f0734069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.071+0000 7f074cc5a700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 0x7f0734069410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07440076e0 tx=0x7f07440097d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.074+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f073804d5a0 con 0x7f074814dba0 2023-10-06T01:40:18.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.257+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0748169f70 con 0x7f074814dba0 2023-10-06T01:40:18.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.257+0000 7f073cff9700 1 -- 172.21.15.98:0/2676058473 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0738051090 con 0x7f074814dba0 2023-10-06T01:40:18.258 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 msgr2=0x7f0734069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 0x7f0734069410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07440076e0 tx=0x7f07440097d0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 msgr2=0x7f074815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07380073f0 tx=0x7f0738001e90 comp rx=0 tx=0).stop 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 shutdown_connections 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0734066f60 0x7f0734069410 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f074815b4f0 0x7f074814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074814d0a0 0x7f07480b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.260 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 --2- 172.21.15.98:0/2676058473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f074814dba0 0x7f074815afb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.261 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 >> 172.21.15.98:0/2676058473 conn(0x7f0748089e90 msgr2=0x7f074815c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.261 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 shutdown_connections 2023-10-06T01:40:18.261 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.259+0000 7f074eebe700 1 -- 172.21.15.98:0/2676058473 wait complete. 2023-10-06T01:40:18.272 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:18.390 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.389+0000 7f2395d4f700 1 Processor -- start 2023-10-06T01:40:18.390 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.389+0000 7f2395d4f700 1 -- start start 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f2395d4f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2390159510 0x7f23900cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f2395d4f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23900cbf20 con 0x7f23900cbae0 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f2395d4f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23900cf610 con 0x7f23901598e0 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f2395d4f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900cf8f0 con 0x7f2390159510 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238f7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2390159510 0x7f23900cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f23900cbae0 0x7f23900c56d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56260/0 (socket says 172.21.15.98:56260) 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238ffff700 1 -- 172.21.15.98:0/4004265733 learned_addr learned my addr 172.21.15.98:0/4004265733 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238f7fe700 1 -- 172.21.15.98:0/4004265733 >> v1:172.21.15.184:6789/0 conn(0x7f23901598e0 legacy=0x7f23900ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238f7fe700 1 -- 172.21.15.98:0/4004265733 >> v1:172.21.15.98:6789/0 conn(0x7f23900cbae0 legacy=0x7f23900c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238f7fe700 1 -- 172.21.15.98:0/4004265733 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23900cfbd0 con 0x7f2390159510 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238f7fe700 1 --2- 172.21.15.98:0/4004265733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2390159510 0x7f23900cb5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f23800098e0 tx=0x7f23800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af90a930c667bc71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.390+0000 7f238e7fc700 1 -- 172.21.15.98:0/4004265733 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380009e80 con 0x7f2390159510 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f238e7fc700 1 -- 172.21.15.98:0/4004265733 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238000b820 con 0x7f2390159510 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f238e7fc700 1 -- 172.21.15.98:0/4004265733 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000bba0 con 0x7f2390159510 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 -- 172.21.15.98:0/4004265733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2390159510 msgr2=0x7f23900cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 --2- 172.21.15.98:0/4004265733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2390159510 0x7f23900cb5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f23800098e0 tx=0x7f23800091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 -- 172.21.15.98:0/4004265733 shutdown_connections 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 --2- 172.21.15.98:0/4004265733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2390159510 0x7f23900cb5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 -- 172.21.15.98:0/4004265733 >> 172.21.15.98:0/4004265733 conn(0x7f2390089e90 msgr2=0x7f23900997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 -- 172.21.15.98:0/4004265733 shutdown_connections 2023-10-06T01:40:18.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.391+0000 7f2395d4f700 1 -- 172.21.15.98:0/4004265733 wait complete. 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.392+0000 7f2395d4f700 1 Processor -- start 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.392+0000 7f2395d4f700 1 -- start start 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23900cbae0 0x7f23900b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390159510 0x7f2390158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900c8440 con 0x7f23901598e0 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900b5b80 con 0x7f2390159510 2023-10-06T01:40:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f2395d4f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23900b5e60 con 0x7f23900cbae0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47072/0 (socket says 172.21.15.98:47072) 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 -- 172.21.15.98:0/2680726907 learned_addr learned my addr 172.21.15.98:0/2680726907 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238f7fe700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23900cbae0 0x7f23900b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238effd700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390159510 0x7f2390158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 -- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23900cbae0 msgr2=0x7f23900b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23900cbae0 0x7f23900b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 -- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390159510 msgr2=0x7f2390158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390159510 0x7f2390158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.393+0000 7f238ffff700 1 -- 172.21.15.98:0/2680726907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380009420 con 0x7f23901598e0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f238ffff700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f238800cc40 tx=0x7f2388001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800a8b0 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800aa10 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23900b6140 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800add0 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.394+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23900b5770 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.395+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23900c8890 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.395+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2388007300 con 0x7f23901598e0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.395+0000 7f23777fe700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.396+0000 7f238f7fe700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.396+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2388084c20 con 0x7f23901598e0 2023-10-06T01:40:18.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.396+0000 7f238f7fe700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 0x7f237c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f23800030c0 tx=0x7f2380016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.400+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23900c8890 con 0x7f23901598e0 2023-10-06T01:40:18.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.583+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23900c8890 con 0x7f23901598e0 2023-10-06T01:40:18.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.583+0000 7f23777fe700 1 -- 172.21.15.98:0/2680726907 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23900c8890 con 0x7f23901598e0 2023-10-06T01:40:18.584 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 msgr2=0x7f237c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 0x7f237c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f23800030c0 tx=0x7f2380016000 comp rx=0 tx=0).stop 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 msgr2=0x7f23900d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f238800cc40 tx=0x7f2388001a50 comp rx=0 tx=0).stop 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 shutdown_connections 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23901598e0 0x7f23900d04e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23900cbae0 0x7f23900b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 --2- 172.21.15.98:0/2680726907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390159510 0x7f2390158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 >> 172.21.15.98:0/2680726907 conn(0x7f2390089e90 msgr2=0x7f2390068710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 shutdown_connections 2023-10-06T01:40:18.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.585+0000 7f2395d4f700 1 -- 172.21.15.98:0/2680726907 wait complete. 2023-10-06T01:40:18.597 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:18.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 Processor -- start 2023-10-06T01:40:18.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 -- start start 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36b4150820 0x7f36b4157530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 0x7f36b406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36b406b900 0x7f36b406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40c7d10 con 0x7f36b406b900 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b406bda0 con 0x7f36b4150820 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.714+0000 7f36ba3d5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b406c080 con 0x7f36b406b4b0 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36b406b900 0x7f36b406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.717 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36b406b900 0x7f36b406ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47080/0 (socket says 172.21.15.98:47080) 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 -- 172.21.15.98:0/169069667 learned_addr learned my addr 172.21.15.98:0/169069667 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b3fff700 1 --2- 172.21.15.98:0/169069667 >> v2:172.21.15.184:3300/0 conn(0x7f36b4150820 0x7f36b4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b37fe700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 0x7f36b406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 -- 172.21.15.98:0/169069667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 msgr2=0x7f36b406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 0x7f36b406e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 -- 172.21.15.98:0/169069667 >> v2:172.21.15.184:3300/0 conn(0x7f36b4150820 msgr2=0x7f36b4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 --2- 172.21.15.98:0/169069667 >> v2:172.21.15.184:3300/0 conn(0x7f36b4150820 0x7f36b4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 -- 172.21.15.98:0/169069667 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40c79a0 con 0x7f36b406b900 2023-10-06T01:40:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b37fe700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 0x7f36b406e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b8972700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b406b900 0x7f36b406ea00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36b406f610 tx=0x7f36ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3668bf9dd455f8c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b2ffd700 1 -- 172.21.15.98:0/169069667 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00e070 con 0x7f36b406b900 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b2ffd700 1 -- 172.21.15.98:0/169069667 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00ba20 con 0x7f36b406b900 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.715+0000 7f36b2ffd700 1 -- 172.21.15.98:0/169069667 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0092e0 con 0x7f36b406b900 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 -- 172.21.15.98:0/169069667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b406b900 msgr2=0x7f36b406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b406b900 0x7f36b406ea00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36b406f610 tx=0x7f36ac009840 comp rx=0 tx=0).stop 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 -- 172.21.15.98:0/169069667 shutdown_connections 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/169069667 >> v2:172.21.15.184:3300/0 conn(0x7f36b4150820 0x7f36b4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b406b900 0x7f36b406ea00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/169069667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b4b0 0x7f36b406e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 -- 172.21.15.98:0/169069667 >> 172.21.15.98:0/169069667 conn(0x7f36b40691d0 msgr2=0x7f36b40cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 -- 172.21.15.98:0/169069667 shutdown_connections 2023-10-06T01:40:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.716+0000 7f36ba3d5700 1 -- 172.21.15.98:0/169069667 wait complete. 2023-10-06T01:40:18.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.717+0000 7f36ba3d5700 1 Processor -- start 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.717+0000 7f36ba3d5700 1 -- start start 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36b406b4b0 0x7f36b414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b900 0x7f36b40aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4167230 con 0x7f36b4150820 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4166690 con 0x7f36b406b4b0 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36ba3d5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4166900 con 0x7f36b406b900 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47082/0 (socket says 172.21.15.98:47082) 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 -- 172.21.15.98:0/2059531464 learned_addr learned my addr 172.21.15.98:0/2059531464 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b3fff700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36b406b4b0 0x7f36b414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b37fe700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b900 0x7f36b40aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 -- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b900 msgr2=0x7f36b40aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b900 0x7f36b40aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 -- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36b406b4b0 msgr2=0x7f36b414f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36b406b4b0 0x7f36b414f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 -- 172.21.15.98:0/2059531464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009ae0 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.718+0000 7f36b8972700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36ac012000 tx=0x7f36ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.719+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac020070 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.719+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac0032f0 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.719+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b4166be0 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.719+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0256c0 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.719+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b4169760 con 0x7f36b4150820 2023-10-06T01:40:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.720+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b41673a0 con 0x7f36b4150820 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.720+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36ac003040 con 0x7f36b4150820 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.720+0000 7f369bfff700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.721+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f36ac087c10 con 0x7f36b4150820 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.721+0000 7f36b3fff700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.721+0000 7f36b3fff700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 0x7f36a00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36b40c2d20 tx=0x7f36a40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.725+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36ac054ae0 con 0x7f36b4150820 2023-10-06T01:40:18.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.903+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36b4064ad0 con 0x7f36b4150820 2023-10-06T01:40:18.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.903+0000 7f369bfff700 1 -- 172.21.15.98:0/2059531464 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36ac058100 con 0x7f36b4150820 2023-10-06T01:40:18.904 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:18.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 msgr2=0x7f36a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 0x7f36a00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36b40c2d20 tx=0x7f36a40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 msgr2=0x7f36b40ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36ac012000 tx=0x7f36ac018000 comp rx=0 tx=0).stop 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 shutdown_connections 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36b4150820 0x7f36b40ab4e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36b406b4b0 0x7f36b414f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 --2- 172.21.15.98:0/2059531464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36b406b900 0x7f36b40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 >> 172.21.15.98:0/2059531464 conn(0x7f36b40691d0 msgr2=0x7f36b40ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 shutdown_connections 2023-10-06T01:40:18.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:18.905+0000 7f36ba3d5700 1 -- 172.21.15.98:0/2059531464 wait complete. 2023-10-06T01:40:18.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:19.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.031+0000 7fd7982f2700 1 Processor -- start 2023-10-06T01:40:19.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.031+0000 7fd7982f2700 1 -- start start 2023-10-06T01:40:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 0x7fd7900aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7900b49e0 0x7fd7900aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b07a0 0x7fd7900b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7900b0ca0 con 0x7fd7900b49e0 2023-10-06T01:40:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7900b0f80 con 0x7fd7901595a0 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd7982f2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900b1260 con 0x7fd7900b07a0 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7900b49e0 0x7fd7900aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7900b49e0 0x7fd7900aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47088/0 (socket says 172.21.15.98:47088) 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 -- 172.21.15.98:0/255368524 learned_addr learned my addr 172.21.15.98:0/255368524 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79608e700 1 --2- 172.21.15.98:0/255368524 >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 0x7fd7900aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79688f700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b07a0 0x7fd7900b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 -- 172.21.15.98:0/255368524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b07a0 msgr2=0x7fd7900b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b07a0 0x7fd7900b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 -- 172.21.15.98:0/255368524 >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 msgr2=0x7fd7900aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 --2- 172.21.15.98:0/255368524 >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 0x7fd7900aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79588d700 1 -- 172.21.15.98:0/255368524 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900b1540 con 0x7fd7900b49e0 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.032+0000 7fd79608e700 1 --2- 172.21.15.98:0/255368524 >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 0x7fd7900aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd79588d700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b49e0 0x7fd7900aaab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd7800099b0 tx=0x7fd780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b036159aa2d13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd79508c700 1 -- 172.21.15.98:0/255368524 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000b820 con 0x7fd7900b49e0 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd79508c700 1 -- 172.21.15.98:0/255368524 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000b980 con 0x7fd7900b49e0 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd79508c700 1 -- 172.21.15.98:0/255368524 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000bd20 con 0x7fd7900b49e0 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 -- 172.21.15.98:0/255368524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b49e0 msgr2=0x7fd7900aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b49e0 0x7fd7900aaab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd7800099b0 tx=0x7fd780009250 comp rx=0 tx=0).stop 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 -- 172.21.15.98:0/255368524 shutdown_connections 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 --2- 172.21.15.98:0/255368524 >> v2:172.21.15.184:3300/0 conn(0x7fd7901595a0 0x7fd7900aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b07a0 0x7fd7900b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 --2- 172.21.15.98:0/255368524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b49e0 0x7fd7900aaab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 -- 172.21.15.98:0/255368524 >> 172.21.15.98:0/255368524 conn(0x7fd790099830 msgr2=0x7fd79006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 -- 172.21.15.98:0/255368524 shutdown_connections 2023-10-06T01:40:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.033+0000 7fd7982f2700 1 -- 172.21.15.98:0/255368524 wait complete. 2023-10-06T01:40:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.034+0000 7fd7982f2700 1 Processor -- start 2023-10-06T01:40:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.034+0000 7fd7982f2700 1 -- start start 2023-10-06T01:40:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b49e0 0x7fd79014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 0x7fd790157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79015b000 con 0x7fd7900b07a0 2023-10-06T01:40:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790153250 con 0x7fd7901595a0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd7982f2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79015ab70 con 0x7fd7900b49e0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd79608e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd79608e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47102/0 (socket says 172.21.15.98:47102) 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd79608e700 1 -- 172.21.15.98:0/2742370046 learned_addr learned my addr 172.21.15.98:0/2742370046 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd79688f700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 0x7fd790157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.035+0000 7fd79588d700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b49e0 0x7fd79014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 -- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b49e0 msgr2=0x7fd79014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b49e0 0x7fd79014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 -- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 msgr2=0x7fd790157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 0x7fd790157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 -- 172.21.15.98:0/2742370046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800094f0 con 0x7fd7900b07a0 2023-10-06T01:40:19.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79688f700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 0x7fd790157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd79608e700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00d890 tx=0x7fd78c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c005a00 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c005b60 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79014e1b0 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0125f0 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.036+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79015a240 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.037+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79015ae50 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.037+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd78c0129a0 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.038+0000 7fd7867fc700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 0x7fd7740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.038+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd78c084a60 con 0x7fd7900b07a0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.038+0000 7fd79588d700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 0x7fd7740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.038+0000 7fd79588d700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 0x7fd7740693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd790088e20 tx=0x7fd780017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.042+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd78c051930 con 0x7fd7900b07a0 2023-10-06T01:40:19.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.227+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd790064ad0 con 0x7fd7900b07a0 2023-10-06T01:40:19.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.227+0000 7fd7867fc700 1 -- 172.21.15.98:0/2742370046 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fd78c054f50 con 0x7fd7900b07a0 2023-10-06T01:40:19.228 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:19.228 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:19.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 msgr2=0x7fd7740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 0x7fd7740693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd790088e20 tx=0x7fd780017000 comp rx=0 tx=0).stop 2023-10-06T01:40:19.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 msgr2=0x7fd790153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00d890 tx=0x7fd78c008040 comp rx=0 tx=0).stop 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 shutdown_connections 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd774066f30 0x7fd7740693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7901595a0 0x7fd790157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7900b07a0 0x7fd790153e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 --2- 172.21.15.98:0/2742370046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7900b49e0 0x7fd79014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.229+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 >> 172.21.15.98:0/2742370046 conn(0x7fd790099830 msgr2=0x7fd7901589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.230+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 shutdown_connections 2023-10-06T01:40:19.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.230+0000 7fd7982f2700 1 -- 172.21.15.98:0/2742370046 wait complete. 2023-10-06T01:40:19.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:19.243 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.359+0000 7f0219ff6700 1 Processor -- start 2023-10-06T01:40:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.359+0000 7f0219ff6700 1 -- start start 2023-10-06T01:40:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0219ff6700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0214159510 0x7f02140cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0219ff6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02140cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0219ff6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02140cbef0 con 0x7f02141598e0 2023-10-06T01:40:19.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0219ff6700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02140cf610 con 0x7f0214159510 2023-10-06T01:40:19.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0219ff6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02140cf8f0 con 0x7f02140cbae0 2023-10-06T01:40:19.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0213fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02140cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f02137fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0214159510 0x7f02140cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.360+0000 7f0212ffd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f02141598e0 0x7f02140ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:52630/0 (socket says 172.21.15.98:52630) 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f0212ffd700 1 -- 172.21.15.98:0/3795879582 learned_addr learned my addr 172.21.15.98:0/3795879582 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02137fe700 1 -- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 msgr2=0x7f02140cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02137fe700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02140cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02137fe700 1 -- 172.21.15.98:0/3795879582 >> v1:172.21.15.184:6789/0 conn(0x7f02141598e0 legacy=0x7f02140ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02137fe700 1 -- 172.21.15.98:0/3795879582 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140cfbd0 con 0x7f0214159510 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02127fc700 1 -- 172.21.15.98:0/3795879582 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251120739 0 0) 0x7f02140cbef0 con 0x7f02141598e0 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f0213fff700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02140cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02137fe700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0214159510 0x7f02140cb5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e60 tx=0x7f0204009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30112b8eae72b58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02127fc700 1 -- 172.21.15.98:0/3795879582 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400e070 con 0x7f0214159510 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02127fc700 1 -- 172.21.15.98:0/3795879582 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400bb60 con 0x7f0214159510 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.361+0000 7f02127fc700 1 -- 172.21.15.98:0/3795879582 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204012430 con 0x7f0214159510 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 -- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0214159510 msgr2=0x7f02140cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0214159510 0x7f02140cb5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e60 tx=0x7f0204009780 comp rx=0 tx=0).stop 2023-10-06T01:40:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 -- 172.21.15.98:0/3795879582 shutdown_connections 2023-10-06T01:40:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02140cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 --2- 172.21.15.98:0/3795879582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0214159510 0x7f02140cb5a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 -- 172.21.15.98:0/3795879582 >> 172.21.15.98:0/3795879582 conn(0x7f0214089e90 msgr2=0x7f02140997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 -- 172.21.15.98:0/3795879582 shutdown_connections 2023-10-06T01:40:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.362+0000 7f0219ff6700 1 -- 172.21.15.98:0/3795879582 wait complete. 2023-10-06T01:40:19.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.363+0000 7f0219ff6700 1 Processor -- start 2023-10-06T01:40:19.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.363+0000 7f0219ff6700 1 -- start start 2023-10-06T01:40:19.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02141556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02141598e0 0x7f0214069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021416d3a0 con 0x7f0214159510 2023-10-06T01:40:19.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021416e200 con 0x7f02141598e0 2023-10-06T01:40:19.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0219ff6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021416d5f0 con 0x7f02140cbae0 2023-10-06T01:40:19.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47106/0 (socket says 172.21.15.98:47106) 2023-10-06T01:40:19.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 -- 172.21.15.98:0/2553920196 learned_addr learned my addr 172.21.15.98:0/2553920196 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:19.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0213fff700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02141598e0 0x7f0214069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f02137fe700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02141556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 -- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 msgr2=0x7f02141556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02141556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 -- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02141598e0 msgr2=0x7f0214069450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02141598e0 0x7f0214069450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 -- 172.21.15.98:0/2553920196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009420 con 0x7f0214159510 2023-10-06T01:40:19.375 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.364+0000 7f0212ffd700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f020800c540 tx=0x7f020800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208005b30 con 0x7f0214159510 2023-10-06T01:40:19.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0208005c90 con 0x7f0214159510 2023-10-06T01:40:19.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0204009a20 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208007300 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021416dbb0 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.365+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0214157200 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.366+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f020801b3b0 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.366+0000 7f01fb7fe700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.366+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0208083cf0 con 0x7f0214159510 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.366+0000 7f02137fe700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.367+0000 7f02137fe700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 0x7f02000693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02040097c0 tx=0x7f0204019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.370+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0208088070 con 0x7f0214159510 2023-10-06T01:40:19.554 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.554+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0214064ad0 con 0x7f0214159510 2023-10-06T01:40:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.554+0000 7f01fb7fe700 1 -- 172.21.15.98:0/2553920196 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f02080170a0 con 0x7f0214159510 2023-10-06T01:40:19.555 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 msgr2=0x7f02000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 0x7f02000693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02040097c0 tx=0x7f0204019000 comp rx=0 tx=0).stop 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 msgr2=0x7f021413d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f020800c540 tx=0x7f020800c850 comp rx=0 tx=0).stop 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 shutdown_connections 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0200066f30 0x7f02000693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02141598e0 0x7f0214069450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02140cbae0 0x7f02141556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 --2- 172.21.15.98:0/2553920196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0214159510 0x7f021413d960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 >> 172.21.15.98:0/2553920196 conn(0x7f0214089e90 msgr2=0x7f021415bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 shutdown_connections 2023-10-06T01:40:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.556+0000 7f0219ff6700 1 -- 172.21.15.98:0/2553920196 wait complete. 2023-10-06T01:40:19.568 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:19.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.684+0000 7fb2cbbe4700 1 Processor -- start 2023-10-06T01:40:19.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 -- start start 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2c4159510 0x7fb2c40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb2c40cbae0 0x7fb2c40cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c40cbf20 con 0x7fb2c41598e0 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c40cf610 con 0x7fb2c40cbae0 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.685+0000 7fb2cbbe4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c40cf8f0 con 0x7fb2c4159510 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb2c40cbae0 0x7fb2c40cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb2c40cbae0 0x7fb2c40cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47120/0 (socket says 172.21.15.98:47120) 2023-10-06T01:40:19.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 -- 172.21.15.98:0/2565121221 learned_addr learned my addr 172.21.15.98:0/2565121221 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2c9980700 1 --2- 172.21.15.98:0/2565121221 >> v2:172.21.15.184:3300/0 conn(0x7fb2c4159510 0x7fb2c40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 -- 172.21.15.98:0/2565121221 >> v2:172.21.15.184:3300/0 conn(0x7fb2c4159510 msgr2=0x7fb2c40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 --2- 172.21.15.98:0/2565121221 >> v2:172.21.15.184:3300/0 conn(0x7fb2c4159510 0x7fb2c40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 -- 172.21.15.98:0/2565121221 >> v1:172.21.15.184:6789/0 conn(0x7fb2c41598e0 legacy=0x7fb2c40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 -- 172.21.15.98:0/2565121221 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40cfbd0 con 0x7fb2c40cbae0 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2ca181700 1 --2- 172.21.15.98:0/2565121221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c40cbae0 0x7fb2c40cef30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4088fb0 tx=0x7fb2bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce553953c0fbb0c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2c897e700 1 -- 172.21.15.98:0/2565121221 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00b860 con 0x7fb2c40cbae0 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2c897e700 1 -- 172.21.15.98:0/2565121221 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00b9c0 con 0x7fb2c40cbae0 2023-10-06T01:40:19.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.686+0000 7fb2c897e700 1 -- 172.21.15.98:0/2565121221 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00bd60 con 0x7fb2c40cbae0 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/2565121221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c40cbae0 msgr2=0x7fb2c40cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/2565121221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c40cbae0 0x7fb2c40cef30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4088fb0 tx=0x7fb2bc009250 comp rx=0 tx=0).stop 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/2565121221 shutdown_connections 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/2565121221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c40cbae0 0x7fb2c40cef30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/2565121221 >> v2:172.21.15.184:3300/0 conn(0x7fb2c4159510 0x7fb2c40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/2565121221 >> 172.21.15.98:0/2565121221 conn(0x7fb2c4089e90 msgr2=0x7fb2c40997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/2565121221 shutdown_connections 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.687+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/2565121221 wait complete. 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.688+0000 7fb2cbbe4700 1 Processor -- start 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.688+0000 7fb2cbbe4700 1 -- start start 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c40cbae0 0x7fb2c4149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c4159510 0x7fb2c415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415cbf0 con 0x7fb2c41598e0 2023-10-06T01:40:19.693 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415b3b0 con 0x7fb2c4159510 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415aa80 con 0x7fb2c40cbae0 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47136/0 (socket says 172.21.15.98:47136) 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 -- 172.21.15.98:0/802696706 learned_addr learned my addr 172.21.15.98:0/802696706 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2c917f700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c4159510 0x7fb2c415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2c9980700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c40cbae0 0x7fb2c4149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 -- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c40cbae0 msgr2=0x7fb2c4149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c40cbae0 0x7fb2c4149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 -- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c4159510 msgr2=0x7fb2c415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c4159510 0x7fb2c415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 -- 172.21.15.98:0/802696706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0094f0 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2ca181700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009290 tx=0x7fb2bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00b860 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc01a630 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c415ad60 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.689+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc01a9f0 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.690+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c415a740 con 0x7fb2c41598e0 2023-10-06T01:40:19.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.690+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c415b520 con 0x7fb2c41598e0 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.691+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2bc02d440 con 0x7fb2c41598e0 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.691+0000 7fb2c1ffb700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 0x7fb2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.691+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb2bc086640 con 0x7fb2c41598e0 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.691+0000 7fb2c9980700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 0x7fb2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.692+0000 7fb2c9980700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 0x7fb2b00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b4009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.696+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2bc0533c0 con 0x7fb2c41598e0 2023-10-06T01:40:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.878+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2c4064ad0 con 0x7fb2c41598e0 2023-10-06T01:40:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.878+0000 7fb2c1ffb700 1 -- 172.21.15.98:0/802696706 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2bc0569e0 con 0x7fb2c41598e0 2023-10-06T01:40:19.879 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 msgr2=0x7fb2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 0x7fb2b00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b4009880 comp rx=0 tx=0).stop 2023-10-06T01:40:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 msgr2=0x7fb2c415c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009290 tx=0x7fb2bc017000 comp rx=0 tx=0).stop 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 shutdown_connections 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb2b0066f30 0x7fb2b00693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2c41598e0 0x7fb2c415c290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c40cbae0 0x7fb2c4149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 --2- 172.21.15.98:0/802696706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c4159510 0x7fb2c415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 >> 172.21.15.98:0/802696706 conn(0x7fb2c4089e90 msgr2=0x7fb2c40cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 shutdown_connections 2023-10-06T01:40:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:19.880+0000 7fb2cbbe4700 1 -- 172.21.15.98:0/802696706 wait complete. 2023-10-06T01:40:19.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:20.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.010+0000 7fbb3464a700 1 Processor -- start 2023-10-06T01:40:20.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.010+0000 7fbb3464a700 1 -- start start 2023-10-06T01:40:20.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.010+0000 7fbb3464a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb2c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb3464a700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0cbf00 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb3464a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0cf610 con 0x7fbb2c1598e0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb3464a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cf8f0 con 0x7fbb2c159510 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb32be7700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbb2c0cbae0 0x7fbb2c0c56d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56276/0 (socket says 172.21.15.98:56276) 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb32be7700 1 -- 172.21.15.98:0/2856680651 learned_addr learned my addr 172.21.15.98:0/2856680651 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb323e6700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb2c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006588251 0 0) 0x7fbb2c0cbf00 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb14003570 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317649678 0 0) 0x7fbb2c0cf610 con 0x7fbb2c1598e0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2c0cbf00 con 0x7fbb2c1598e0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328852305 0 0) 0x7fbb14003570 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 msgr2=0x7fbb2c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb2c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 >> v1:172.21.15.184:6789/0 conn(0x7fbb2c1598e0 legacy=0x7fbb2c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0cfbd0 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fbb24001470 con 0x7fbb2c0cbae0 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb323e6700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb2c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 >> v1:172.21.15.98:6789/0 conn(0x7fbb2c0cbae0 legacy=0x7fbb2c0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb14003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14004d50 0x7fbb140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cf8f0 con 0x7fbb14004160 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400bba0 con 0x7fbb14004d50 2023-10-06T01:40:20.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400bd60 con 0x7fbb2c159510 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb31be5700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.011+0000 7fbb323e6700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb14003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb32be7700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14004d50 0x7fbb140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 -- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 msgr2=0x7fbb14003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c159510 0x7fbb14003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 -- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14004d50 msgr2=0x7fbb140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb14004d50 0x7fbb140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 -- 172.21.15.98:0/2856680651 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c009a20 con 0x7fbb14004160 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb31be5700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb14004810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb200019e0 tx=0x7fbb20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42754b6f3b82e5da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000c5d0 con 0x7fbb14004160 2023-10-06T01:40:20.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000c730 con 0x7fbb14004160 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb313e4700 1 -- 172.21.15.98:0/2856680651 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000cad0 con 0x7fbb14004160 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb3464a700 1 -- 172.21.15.98:0/2856680651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 msgr2=0x7fbb14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb3464a700 1 --2- 172.21.15.98:0/2856680651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb14004810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb200019e0 tx=0x7fbb20007d30 comp rx=0 tx=0).stop 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.012+0000 7fbb32be7700 1 -- 172.21.15.98:0/2856680651 reap_dead start 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.013+0000 7fbb3464a700 1 -- 172.21.15.98:0/2856680651 shutdown_connections 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.013+0000 7fbb3464a700 1 -- 172.21.15.98:0/2856680651 >> 172.21.15.98:0/2856680651 conn(0x7fbb2c089e90 msgr2=0x7fbb2c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.013+0000 7fbb3464a700 1 -- 172.21.15.98:0/2856680651 shutdown_connections 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.013+0000 7fbb3464a700 1 -- 172.21.15.98:0/2856680651 wait complete. 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 Processor -- start 2023-10-06T01:40:20.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 -- start start 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c169ad0 0x7fbb2c16db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0c8920 0x7fbb2c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c16e060 con 0x7fbb14004160 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0b32f0 con 0x7fbb2c169ad0 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb3464a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c16a010 con 0x7fbb2c0c8920 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb323e6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb323e6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47150/0 (socket says 172.21.15.98:47150) 2023-10-06T01:40:20.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.014+0000 7fbb323e6700 1 -- 172.21.15.98:0/897508620 learned_addr learned my addr 172.21.15.98:0/897508620 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb31be5700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c169ad0 0x7fbb2c16db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb32be7700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0c8920 0x7fbb2c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 -- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0c8920 msgr2=0x7fbb2c0c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0c8920 0x7fbb2c0c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 -- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c169ad0 msgr2=0x7fbb2c16db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c169ad0 0x7fbb2c16db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 -- 172.21.15.98:0/897508620 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c16a2f0 con 0x7fbb14004160 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb323e6700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00f2e0 tx=0x7fbb1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00e040 con 0x7fbb14004160 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00f040 con 0x7fbb14004160 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c16a5d0 con 0x7fbb14004160 2023-10-06T01:40:20.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c015680 con 0x7fbb14004160 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.015+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c14d220 con 0x7fbb14004160 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.016+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c064ad0 con 0x7fbb14004160 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.016+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb1c015a30 con 0x7fbb14004160 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.017+0000 7fbb2a7fc700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.017+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbb1c085700 con 0x7fbb14004160 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.017+0000 7fbb31be5700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.017+0000 7fbb31be5700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 0x7fbb180693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c155290 tx=0x7fbb20015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.021+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb1c052520 con 0x7fbb14004160 2023-10-06T01:40:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.204+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb2c0b35d0 con 0x7fbb14004160 2023-10-06T01:40:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.204+0000 7fbb2a7fc700 1 -- 172.21.15.98:0/897508620 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb1c05b020 con 0x7fbb14004160 2023-10-06T01:40:20.205 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 msgr2=0x7fbb180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 0x7fbb180693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c155290 tx=0x7fbb20015000 comp rx=0 tx=0).stop 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 msgr2=0x7fbb2c14c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00f2e0 tx=0x7fbb1c00b040 comp rx=0 tx=0).stop 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 shutdown_connections 2023-10-06T01:40:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c0c8920 0x7fbb2c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb14004160 0x7fbb2c14c9a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 --2- 172.21.15.98:0/897508620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c169ad0 0x7fbb2c16db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 >> 172.21.15.98:0/897508620 conn(0x7fbb2c089e90 msgr2=0x7fbb2c15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 shutdown_connections 2023-10-06T01:40:20.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.206+0000 7fbb3464a700 1 -- 172.21.15.98:0/897508620 wait complete. 2023-10-06T01:40:20.218 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:20.309 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:40:20.313 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:40:20.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.338+0000 7f44d9cc3700 1 Processor -- start 2023-10-06T01:40:20.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.338+0000 7f44d9cc3700 1 -- start start 2023-10-06T01:40:20.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d9cc3700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f44d4159510 0x7f44d40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d9cc3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d41598e0 0x7f44d40cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d9cc3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d40cf370 con 0x7f44d40ce820 2023-10-06T01:40:20.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d9cc3700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40cf610 con 0x7f44d41598e0 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d9cc3700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40cf8f0 con 0x7f44d4159510 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d41598e0 0x7f44d40cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d41598e0 0x7f44d40cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47156/0 (socket says 172.21.15.98:47156) 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 -- 172.21.15.98:0/2617212154 learned_addr learned my addr 172.21.15.98:0/2617212154 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d37fe700 1 --2- 172.21.15.98:0/2617212154 >> v2:172.21.15.184:3300/0 conn(0x7f44d4159510 0x7f44d40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 -- 172.21.15.98:0/2617212154 >> v2:172.21.15.184:3300/0 conn(0x7f44d4159510 msgr2=0x7f44d40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 --2- 172.21.15.98:0/2617212154 >> v2:172.21.15.184:3300/0 conn(0x7f44d4159510 0x7f44d40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 -- 172.21.15.98:0/2617212154 >> v1:172.21.15.184:6789/0 conn(0x7f44d40ce820 legacy=0x7f44d40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 -- 172.21.15.98:0/2617212154 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40cfbd0 con 0x7f44d41598e0 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.339+0000 7f44d2ffd700 1 --2- 172.21.15.98:0/2617212154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d41598e0 0x7f44d40cbae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44d4088fb0 tx=0x7f44c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1684b6a86bfff0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d27fc700 1 -- 172.21.15.98:0/2617212154 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c8009e70 con 0x7f44d41598e0 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d27fc700 1 -- 172.21.15.98:0/2617212154 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800b860 con 0x7f44d41598e0 2023-10-06T01:40:20.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d27fc700 1 -- 172.21.15.98:0/2617212154 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800bc00 con 0x7f44d41598e0 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 -- 172.21.15.98:0/2617212154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d41598e0 msgr2=0x7f44d40cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/2617212154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d41598e0 0x7f44d40cbae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44d4088fb0 tx=0x7f44c8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 -- 172.21.15.98:0/2617212154 shutdown_connections 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/2617212154 >> v2:172.21.15.184:3300/0 conn(0x7f44d4159510 0x7f44d40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/2617212154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d41598e0 0x7f44d40cbae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 -- 172.21.15.98:0/2617212154 >> 172.21.15.98:0/2617212154 conn(0x7f44d4089e90 msgr2=0x7f44d40997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 -- 172.21.15.98:0/2617212154 shutdown_connections 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.340+0000 7f44d9cc3700 1 -- 172.21.15.98:0/2617212154 wait complete. 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.341+0000 7f44d9cc3700 1 Processor -- start 2023-10-06T01:40:20.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.341+0000 7f44d9cc3700 1 -- start start 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d40ce820 0x7f44d4167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d41598e0 0x7f44d40c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d4153250 con 0x7f44d4159510 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d416c610 con 0x7f44d41598e0 2023-10-06T01:40:20.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d9cc3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d41684b0 con 0x7f44d40ce820 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47166/0 (socket says 172.21.15.98:47166) 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 -- 172.21.15.98:0/1742746059 learned_addr learned my addr 172.21.15.98:0/1742746059 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d3fff700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d41598e0 0x7f44d40c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d37fe700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d40ce820 0x7f44d4167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 -- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d40ce820 msgr2=0x7f44d4167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d40ce820 0x7f44d4167f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 -- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d41598e0 msgr2=0x7f44d40c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d41598e0 0x7f44d40c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.342+0000 7f44d2ffd700 1 -- 172.21.15.98:0/1742746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80094f0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44d2ffd700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44c8006fd0 tx=0x7f44c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800bbe0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c8011420 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d416b2c0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c80117c0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.343+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d416bfc0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.344+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d416c7c0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.344+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44c8011bf0 con 0x7f44d4159510 2023-10-06T01:40:20.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.345+0000 7f44bb7fe700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.345+0000 7f44d37fe700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.345+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f44c80861e0 con 0x7f44d4159510 2023-10-06T01:40:20.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.345+0000 7f44d37fe700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 0x7f44c00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009db0 tx=0x7f44c40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.349+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44d416c7c0 con 0x7f44d4159510 2023-10-06T01:40:20.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.531+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44d416c7c0 con 0x7f44d4159510 2023-10-06T01:40:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.531+0000 7f44bb7fe700 1 -- 172.21.15.98:0/1742746059 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44d416c7c0 con 0x7f44d4159510 2023-10-06T01:40:20.532 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 msgr2=0x7f44c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 0x7f44c00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009db0 tx=0x7f44c40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 msgr2=0x7f44d416ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44c8006fd0 tx=0x7f44c8017000 comp rx=0 tx=0).stop 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 shutdown_connections 2023-10-06T01:40:20.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d41598e0 0x7f44d40c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d40ce820 0x7f44d4167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 --2- 172.21.15.98:0/1742746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d4159510 0x7f44d416ad80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 >> 172.21.15.98:0/1742746059 conn(0x7f44d4089e90 msgr2=0x7f44d414c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 shutdown_connections 2023-10-06T01:40:20.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.533+0000 7f44d9cc3700 1 -- 172.21.15.98:0/1742746059 wait complete. 2023-10-06T01:40:20.546 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:20.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.663+0000 7f76253cb700 1 Processor -- start 2023-10-06T01:40:20.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.663+0000 7f76253cb700 1 -- start start 2023-10-06T01:40:20.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f76253cb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76201539d0 0x7f762015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f76253cb700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f76200d0a00 0x7f76200ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f76253cb700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76200cdc30 con 0x7f76200ccab0 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f76253cb700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200cb500 con 0x7f76200d0a00 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f76253cb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762015a8f0 con 0x7f76201539d0 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f7624bca700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f76200d0a00 0x7f76200ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f761f7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f76200ccab0 0x7f76200d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56284/0 (socket says 172.21.15.98:56284) 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f7624bca700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f76200d0a00 0x7f76200ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47168/0 (socket says 172.21.15.98:47168) 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f761f7fe700 1 -- 172.21.15.98:0/3471355363 learned_addr learned my addr 172.21.15.98:0/3471355363 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.664+0000 7f761ffff700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76201539d0 0x7f762015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f7624bca700 1 -- 172.21.15.98:0/3471355363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76201539d0 msgr2=0x7f762015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f7624bca700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76201539d0 0x7f762015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f7624bca700 1 -- 172.21.15.98:0/3471355363 >> v1:172.21.15.98:6789/0 conn(0x7f76200ccab0 legacy=0x7f76200d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f7624bca700 1 -- 172.21.15.98:0/3471355363 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200cd020 con 0x7f76200d0a00 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f761effd700 1 -- 172.21.15.98:0/3471355363 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768946379 0 0) 0x7f76200cdc30 con 0x7f76200ccab0 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f7624bca700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200ce8a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7618009e60 tx=0x7f7618009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd278dd96b081c96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f761effd700 1 -- 172.21.15.98:0/3471355363 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800e070 con 0x7f76200d0a00 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f761effd700 1 -- 172.21.15.98:0/3471355363 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761800bb60 con 0x7f76200d0a00 2023-10-06T01:40:20.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f761effd700 1 -- 172.21.15.98:0/3471355363 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618012430 con 0x7f76200d0a00 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 -- 172.21.15.98:0/3471355363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 msgr2=0x7f76200ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200ce8a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7618009e60 tx=0x7f7618009780 comp rx=0 tx=0).stop 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 -- 172.21.15.98:0/3471355363 shutdown_connections 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200ce8a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 --2- 172.21.15.98:0/3471355363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76201539d0 0x7f762015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 -- 172.21.15.98:0/3471355363 >> 172.21.15.98:0/3471355363 conn(0x7f7620068100 msgr2=0x7f76200684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 -- 172.21.15.98:0/3471355363 shutdown_connections 2023-10-06T01:40:20.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.665+0000 7f76253cb700 1 -- 172.21.15.98:0/3471355363 wait complete. 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.667+0000 7f76253cb700 1 Processor -- start 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.667+0000 7f76253cb700 1 -- start start 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.667+0000 7f76253cb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200ccab0 0x7f76200b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.667+0000 7f76253cb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76201539d0 0x7f762015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200c7a60 con 0x7f76200d0a00 2023-10-06T01:40:20.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762015d920 con 0x7f76201539d0 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76200b15d0 con 0x7f76200ccab0 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47182/0 (socket says 172.21.15.98:47182) 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 -- 172.21.15.98:0/2055181171 learned_addr learned my addr 172.21.15.98:0/2055181171 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f7624bca700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76201539d0 0x7f762015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761ffff700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200ccab0 0x7f76200b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 -- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200ccab0 msgr2=0x7f76200b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200ccab0 0x7f76200b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 -- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76201539d0 msgr2=0x7f762015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76201539d0 0x7f762015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 -- 172.21.15.98:0/2055181171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618009a20 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f761f7fe700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f761400c5b0 tx=0x7f7614001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614005b30 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614005c90 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76200b18b0 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614007300 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.668+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76200c76f0 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.669+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200c7bd0 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.670+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f761401b3b0 con 0x7f76200d0a00 2023-10-06T01:40:20.674 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.670+0000 7f7607fff700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.670+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7614083ed0 con 0x7f76200d0a00 2023-10-06T01:40:20.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.670+0000 7f761ffff700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.670+0000 7f761ffff700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 0x7f760c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76201525a0 tx=0x7f7610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.674+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7614050cf0 con 0x7f76200d0a00 2023-10-06T01:40:20.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.856+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f762005f270 con 0x7f76200d0a00 2023-10-06T01:40:20.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.857+0000 7f7607fff700 1 -- 172.21.15.98:0/2055181171 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7614017020 con 0x7f76200d0a00 2023-10-06T01:40:20.858 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:20.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 msgr2=0x7f760c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 0x7f760c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76201525a0 tx=0x7f7610008040 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 msgr2=0x7f76200b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f761400c5b0 tx=0x7f7614001070 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 shutdown_connections 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76201539d0 0x7f762015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200ccab0 0x7f76200b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 --2- 172.21.15.98:0/2055181171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76200d0a00 0x7f76200b0fa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 >> 172.21.15.98:0/2055181171 conn(0x7f7620068100 msgr2=0x7f76200cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 shutdown_connections 2023-10-06T01:40:20.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.859+0000 7f76253cb700 1 -- 172.21.15.98:0/2055181171 wait complete. 2023-10-06T01:40:20.871 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:20.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.984+0000 7f3d1196d700 1 Processor -- start 2023-10-06T01:40:20.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.984+0000 7f3d1196d700 1 -- start start 2023-10-06T01:40:20.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d1196d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d0c069720 0x7f3d0c069af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d1196d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d0c0adaa0 0x7f3d0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d1196d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0c06d8d0 con 0x7f3d0c0aeef0 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d1196d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c0cd040 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d1196d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c06e710 con 0x7f3d0c069720 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d027fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d0c0adaa0 0x7f3d0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d027fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d0c0adaa0 0x7f3d0c06d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47190/0 (socket says 172.21.15.98:47190) 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.985+0000 7f3d027fc700 1 -- 172.21.15.98:0/2370238334 learned_addr learned my addr 172.21.15.98:0/2370238334 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d0affd700 1 --2- 172.21.15.98:0/2370238334 >> v2:172.21.15.184:3300/0 conn(0x7f3d0c069720 0x7f3d0c069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d027fc700 1 -- 172.21.15.98:0/2370238334 >> v2:172.21.15.184:3300/0 conn(0x7f3d0c069720 msgr2=0x7f3d0c069af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d027fc700 1 --2- 172.21.15.98:0/2370238334 >> v2:172.21.15.184:3300/0 conn(0x7f3d0c069720 0x7f3d0c069af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d027fc700 1 -- 172.21.15.98:0/2370238334 >> v1:172.21.15.184:6789/0 conn(0x7f3d0c0aeef0 legacy=0x7f3d0c06eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d027fc700 1 -- 172.21.15.98:0/2370238334 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c06db00 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d027fc700 1 --2- 172.21.15.98:0/2370238334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c06d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c070190 tx=0x7f3cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf0e411e257e503 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d0a7fc700 1 -- 172.21.15.98:0/2370238334 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400b860 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d0a7fc700 1 -- 172.21.15.98:0/2370238334 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf400b9c0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d0a7fc700 1 -- 172.21.15.98:0/2370238334 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400bd60 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 -- 172.21.15.98:0/2370238334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 msgr2=0x7f3d0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 --2- 172.21.15.98:0/2370238334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c06d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c070190 tx=0x7f3cf4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 -- 172.21.15.98:0/2370238334 shutdown_connections 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 --2- 172.21.15.98:0/2370238334 >> v2:172.21.15.184:3300/0 conn(0x7f3d0c069720 0x7f3d0c069af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 --2- 172.21.15.98:0/2370238334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c06d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.986+0000 7f3d1196d700 1 -- 172.21.15.98:0/2370238334 >> 172.21.15.98:0/2370238334 conn(0x7f3d0c0680c0 msgr2=0x7f3d0c0cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:20.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.987+0000 7f3d1196d700 1 -- 172.21.15.98:0/2370238334 shutdown_connections 2023-10-06T01:40:20.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.987+0000 7f3d1196d700 1 -- 172.21.15.98:0/2370238334 wait complete. 2023-10-06T01:40:20.992 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 Processor -- start 2023-10-06T01:40:20.992 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 -- start start 2023-10-06T01:40:20.992 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 0x7f3d0c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d0c0aeef0 0x7f3d0c0c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1579f0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0c0de0 con 0x7f3d0c069720 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.988+0000 7f3d1196d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0c10c0 con 0x7f3d0c0aeef0 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47204/0 (socket says 172.21.15.98:47204) 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 -- 172.21.15.98:0/1041605930 learned_addr learned my addr 172.21.15.98:0/1041605930 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d0affd700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 0x7f3d0c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d0b7fe700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d0c0aeef0 0x7f3d0c0c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 -- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d0c0aeef0 msgr2=0x7f3d0c0c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d0c0aeef0 0x7f3d0c0c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 -- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 msgr2=0x7f3d0c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 0x7f3d0c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 -- 172.21.15.98:0/1041605930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40094f0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d0affd700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 0x7f3d0c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d027fc700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009290 tx=0x7f3cf4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4011e60 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf401a630 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf401a9f0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c0c13a0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.989+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c167930 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.990+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c064ad0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.990+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cf402d440 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.991+0000 7f3d037fe700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 0x7f3cec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.991+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3cf40865b0 con 0x7f3d0c0adaa0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.991+0000 7f3d0affd700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 0x7f3cec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.991+0000 7f3d0affd700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 0x7f3cec0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006fd0 tx=0x7f3cfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:20.995 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:20.995+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf4053450 con 0x7f3d0c0adaa0 2023-10-06T01:40:21.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.180+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d0c06d110 con 0x7f3d0c0adaa0 2023-10-06T01:40:21.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.181+0000 7f3d037fe700 1 -- 172.21.15.98:0/1041605930 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f3cf4056a70 con 0x7f3d0c0adaa0 2023-10-06T01:40:21.182 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:21.182 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:21.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 msgr2=0x7f3cec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 0x7f3cec0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006fd0 tx=0x7f3cfc008040 comp rx=0 tx=0).stop 2023-10-06T01:40:21.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 msgr2=0x7f3d0c1574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009290 tx=0x7f3cf4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:21.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 shutdown_connections 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3cec066f30 0x7f3cec0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d0c0aeef0 0x7f3d0c0c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c069720 0x7f3d0c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 --2- 172.21.15.98:0/1041605930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0c0adaa0 0x7f3d0c1574b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 >> 172.21.15.98:0/1041605930 conn(0x7f3d0c0680c0 msgr2=0x7f3d0c0b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 shutdown_connections 2023-10-06T01:40:21.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.183+0000 7f3d1196d700 1 -- 172.21.15.98:0/1041605930 wait complete. 2023-10-06T01:40:21.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:21.196 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:21.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.313+0000 7ff1a9cda700 1 Processor -- start 2023-10-06T01:40:21.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.313+0000 7ff1a9cda700 1 -- start start 2023-10-06T01:40:21.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a9cda700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 0x7ff1a4069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a9cda700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff1a415a6f0 0x7ff1a4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a9cda700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a415a260 con 0x7ff1a41596f0 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a9cda700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a406ace0 con 0x7ff1a415a6f0 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a9cda700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4159c10 con 0x7ff1a4159320 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a8cd8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 0x7ff1a4069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff1a415a6f0 0x7ff1a4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a94d9700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff1a41596f0 0x7ff1a415cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:52668/0 (socket says 172.21.15.98:52668) 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a94d9700 1 -- 172.21.15.98:0/3753001498 learned_addr learned my addr 172.21.15.98:0/3753001498 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a3fff700 1 -- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 msgr2=0x7ff1a4069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a3fff700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 0x7ff1a4069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a3fff700 1 -- 172.21.15.98:0/3753001498 >> v1:172.21.15.184:6789/0 conn(0x7ff1a41596f0 legacy=0x7ff1a415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.314+0000 7ff1a37fe700 1 -- 172.21.15.98:0/3753001498 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291712531 0 0) 0x7ff1a415a260 con 0x7ff1a41596f0 2023-10-06T01:40:21.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a3fff700 1 -- 172.21.15.98:0/3753001498 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4159ef0 con 0x7ff1a415a6f0 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a8cd8700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 0x7ff1a4069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a3fff700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a415a6f0 0x7ff1a4069f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098e0 tx=0x7ff1900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cd13f76081ce2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a37fe700 1 -- 172.21.15.98:0/3753001498 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000b640 con 0x7ff1a415a6f0 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a37fe700 1 -- 172.21.15.98:0/3753001498 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19000b7a0 con 0x7ff1a415a6f0 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a37fe700 1 -- 172.21.15.98:0/3753001498 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000bb40 con 0x7ff1a415a6f0 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 -- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a415a6f0 msgr2=0x7ff1a4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a415a6f0 0x7ff1a4069f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098e0 tx=0x7ff1900091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 -- 172.21.15.98:0/3753001498 shutdown_connections 2023-10-06T01:40:21.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a4159320 0x7ff1a4069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/3753001498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a415a6f0 0x7ff1a4069f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 -- 172.21.15.98:0/3753001498 >> 172.21.15.98:0/3753001498 conn(0x7ff1a40cd220 msgr2=0x7ff1a40c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.315+0000 7ff1a9cda700 1 -- 172.21.15.98:0/3753001498 shutdown_connections 2023-10-06T01:40:21.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.316+0000 7ff1a9cda700 1 -- 172.21.15.98:0/3753001498 wait complete. 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 Processor -- start 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 -- start start 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a4159320 0x7ff1a415d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a415a6f0 0x7ff1a4154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41538b0 con 0x7ff1a41596f0 2023-10-06T01:40:21.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4153b90 con 0x7ff1a4159320 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.317+0000 7ff1a9cda700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4153e70 con 0x7ff1a415a6f0 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47210/0 (socket says 172.21.15.98:47210) 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 -- 172.21.15.98:0/10838774 learned_addr learned my addr 172.21.15.98:0/10838774 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a8cd8700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a4159320 0x7ff1a415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a94d9700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a415a6f0 0x7ff1a4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 -- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a415a6f0 msgr2=0x7ff1a4154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a415a6f0 0x7ff1a4154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 -- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a4159320 msgr2=0x7ff1a415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a4159320 0x7ff1a415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 -- 172.21.15.98:0/10838774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff198009420 con 0x7ff1a41596f0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a3fff700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4069590 tx=0x7ff190017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19001f070 con 0x7ff1a41596f0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190002f30 con 0x7ff1a41596f0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff190009420 con 0x7ff1a41596f0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190011420 con 0x7ff1a41596f0 2023-10-06T01:40:21.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.318+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4154430 con 0x7ff1a41596f0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.319+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4154650 con 0x7ff1a41596f0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.319+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff190011850 con 0x7ff1a41596f0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.320+0000 7ff1a0ff9700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 0x7ff1880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.320+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff190085c10 con 0x7ff1a41596f0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.320+0000 7ff1a8cd8700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 0x7ff1880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.320+0000 7ff1a8cd8700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 0x7ff1880693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff198009f40 tx=0x7ff1980091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.324+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff190052a30 con 0x7ff1a41596f0 2023-10-06T01:40:21.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.513+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1a405f270 con 0x7ff1a41596f0 2023-10-06T01:40:21.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.513+0000 7ff1a0ff9700 1 -- 172.21.15.98:0/10838774 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff190056050 con 0x7ff1a41596f0 2023-10-06T01:40:21.514 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:21.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 msgr2=0x7ff1880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 0x7ff1880693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff198009f40 tx=0x7ff1980091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 msgr2=0x7ff1a41452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4069590 tx=0x7ff190017000 comp rx=0 tx=0).stop 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 shutdown_connections 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff188066f30 0x7ff1880693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1a415a6f0 0x7ff1a4154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1a4159320 0x7ff1a415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 --2- 172.21.15.98:0/10838774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a41596f0 0x7ff1a41452c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.515+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 >> 172.21.15.98:0/10838774 conn(0x7ff1a40cd220 msgr2=0x7ff1a40af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.516+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 shutdown_connections 2023-10-06T01:40:21.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.516+0000 7ff1a9cda700 1 -- 172.21.15.98:0/10838774 wait complete. 2023-10-06T01:40:21.528 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:21.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.644+0000 7f50f5f1b700 1 Processor -- start 2023-10-06T01:40:21.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.644+0000 7f50f5f1b700 1 -- start start 2023-10-06T01:40:21.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50f5f1b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f50f0159510 0x7f50f00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50f5f1b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00cbae0 0x7f50f00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50f5f1b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f00cbef0 con 0x7f50f01598e0 2023-10-06T01:40:21.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50f5f1b700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f00cf610 con 0x7f50f0159510 2023-10-06T01:40:21.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50f5f1b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00cf8f0 con 0x7f50f00cbae0 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50ef7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f50f0159510 0x7f50f00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50ef7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f50f0159510 0x7f50f00cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47220/0 (socket says 172.21.15.98:47220) 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.645+0000 7f50ef7fe700 1 -- 172.21.15.98:0/2683545173 learned_addr learned my addr 172.21.15.98:0/2683545173 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ef7fe700 1 -- 172.21.15.98:0/2683545173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00cbae0 msgr2=0x7f50f00cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/2683545173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00cbae0 0x7f50f00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ef7fe700 1 -- 172.21.15.98:0/2683545173 >> v1:172.21.15.184:6789/0 conn(0x7f50f01598e0 legacy=0x7f50f00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ef7fe700 1 -- 172.21.15.98:0/2683545173 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00cfbd0 con 0x7f50f0159510 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/2683545173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00cb5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009f20 tx=0x7f50e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b4e84ea12c0045 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ee7fc700 1 -- 172.21.15.98:0/2683545173 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000e070 con 0x7f50f0159510 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ee7fc700 1 -- 172.21.15.98:0/2683545173 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000ba40 con 0x7f50f0159510 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50ee7fc700 1 -- 172.21.15.98:0/2683545173 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00092e0 con 0x7f50f0159510 2023-10-06T01:40:21.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50f5f1b700 1 -- 172.21.15.98:0/2683545173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 msgr2=0x7f50f00cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.646+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/2683545173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00cb5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009f20 tx=0x7f50e0009840 comp rx=0 tx=0).stop 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 -- 172.21.15.98:0/2683545173 shutdown_connections 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/2683545173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f00cbae0 0x7f50f00cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/2683545173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00cb5a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 -- 172.21.15.98:0/2683545173 >> 172.21.15.98:0/2683545173 conn(0x7f50f0089e90 msgr2=0x7f50f00997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 -- 172.21.15.98:0/2683545173 shutdown_connections 2023-10-06T01:40:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.647+0000 7f50f5f1b700 1 -- 172.21.15.98:0/2683545173 wait complete. 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.648+0000 7f50f5f1b700 1 Processor -- start 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.648+0000 7f50f5f1b700 1 -- start start 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 0x7f50f0169a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f01598e0 0x7f50f00b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00c7a90 con 0x7f50f0159510 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00c7d70 con 0x7f50f00cbae0 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50f5f1b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00c8050 con 0x7f50f01598e0 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47236/0 (socket says 172.21.15.98:47236) 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 -- 172.21.15.98:0/1199113079 learned_addr learned my addr 172.21.15.98:0/1199113079 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 0x7f50f0169a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50effff700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f01598e0 0x7f50f00b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 -- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f01598e0 msgr2=0x7f50f00b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f01598e0 0x7f50f00b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 -- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 msgr2=0x7f50f0169a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 0x7f50f0169a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 -- 172.21.15.98:0/1199113079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009ae0 con 0x7f50f0159510 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 0x7f50f0169a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.649+0000 7f50eeffd700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f50e400b510 tx=0x7f50e40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.650+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400f510 con 0x7f50f0159510 2023-10-06T01:40:21.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.650+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e400f670 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.650+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f00c8330 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.650+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400fa30 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.650+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f00c7600 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.651+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f00c7850 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.651+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50e401a400 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.651+0000 7f50d77fe700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.651+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f50e4084420 con 0x7f50f0159510 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.651+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.652+0000 7f50ef7fe700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 0x7f50dc0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009880 tx=0x7f50e0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.655+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50f00c7850 con 0x7f50f0159510 2023-10-06T01:40:21.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.838+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50f00c7850 con 0x7f50f0159510 2023-10-06T01:40:21.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.839+0000 7f50d77fe700 1 -- 172.21.15.98:0/1199113079 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50f00c7850 con 0x7f50f0159510 2023-10-06T01:40:21.839 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 msgr2=0x7f50dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 0x7f50dc0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009880 tx=0x7f50e0017000 comp rx=0 tx=0).stop 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 msgr2=0x7f50f00b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f50e400b510 tx=0x7f50e40090f0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 shutdown_connections 2023-10-06T01:40:21.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50f01598e0 0x7f50f00b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50f00cbae0 0x7f50f0169a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 --2- 172.21.15.98:0/1199113079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50f0159510 0x7f50f00b5440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 >> 172.21.15.98:0/1199113079 conn(0x7f50f0089e90 msgr2=0x7f50f0068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 shutdown_connections 2023-10-06T01:40:21.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.841+0000 7f50f5f1b700 1 -- 172.21.15.98:0/1199113079 wait complete. 2023-10-06T01:40:21.854 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:21.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.969+0000 7f45dc937700 1 Processor -- start 2023-10-06T01:40:21.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.969+0000 7f45dc937700 1 -- start start 2023-10-06T01:40:21.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.970+0000 7f45dc937700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45d80c4490 0x7f45d8068700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.970+0000 7f45dc937700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45d80cc780 0x7f45d80ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.970+0000 7f45dc937700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80c96b0 con 0x7f45d80b0860 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.970+0000 7f45dc937700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d8159c40 con 0x7f45d80c4490 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.970+0000 7f45dc937700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80c8aa0 con 0x7f45d80cc780 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45d80c4490 0x7f45d8068700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d659c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f45d80b0860 0x7f45d80cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56290/0 (socket says 172.21.15.98:56290) 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d659c700 1 -- 172.21.15.98:0/1174357395 learned_addr learned my addr 172.21.15.98:0/1174357395 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 -- 172.21.15.98:0/1174357395 >> v2:172.21.15.184:3300/0 conn(0x7f45d80cc780 msgr2=0x7f45d80ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d6d9d700 1 --2- 172.21.15.98:0/1174357395 >> v2:172.21.15.184:3300/0 conn(0x7f45d80cc780 0x7f45d80ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 --2- 172.21.15.98:0/1174357395 >> v2:172.21.15.184:3300/0 conn(0x7f45d80cc780 0x7f45d80ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d559a700 1 -- 172.21.15.98:0/1174357395 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327956081 0 0) 0x7f45d80c96b0 con 0x7f45d80b0860 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 -- 172.21.15.98:0/1174357395 >> v1:172.21.15.98:6789/0 conn(0x7f45d80b0860 legacy=0x7f45d80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 -- 172.21.15.98:0/1174357395 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80c8d80 con 0x7f45d80c4490 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d5d9b700 1 --2- 172.21.15.98:0/1174357395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80c4490 0x7f45d8068700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f45d80cde10 tx=0x7f45cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4512181a3ce3ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d559a700 1 -- 172.21.15.98:0/1174357395 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00b6b0 con 0x7f45d80c4490 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d559a700 1 -- 172.21.15.98:0/1174357395 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc00b810 con 0x7f45d80c4490 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.971+0000 7f45d559a700 1 -- 172.21.15.98:0/1174357395 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00bbb0 con 0x7f45d80c4490 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 -- 172.21.15.98:0/1174357395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80c4490 msgr2=0x7f45d8068700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 --2- 172.21.15.98:0/1174357395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80c4490 0x7f45d8068700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f45d80cde10 tx=0x7f45cc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 -- 172.21.15.98:0/1174357395 shutdown_connections 2023-10-06T01:40:21.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 --2- 172.21.15.98:0/1174357395 >> v2:172.21.15.184:3300/0 conn(0x7f45d80cc780 0x7f45d80ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 --2- 172.21.15.98:0/1174357395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80c4490 0x7f45d8068700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 -- 172.21.15.98:0/1174357395 >> 172.21.15.98:0/1174357395 conn(0x7f45d8088000 msgr2=0x7f45d80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:21.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 -- 172.21.15.98:0/1174357395 shutdown_connections 2023-10-06T01:40:21.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.972+0000 7f45dc937700 1 -- 172.21.15.98:0/1174357395 wait complete. 2023-10-06T01:40:21.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.973+0000 7f45dc937700 1 Processor -- start 2023-10-06T01:40:21.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.973+0000 7f45dc937700 1 -- start start 2023-10-06T01:40:21.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45d80c4490 0x7f45d80c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d80cc780 0x7f45d80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80c72d0 con 0x7f45d80b0860 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d815b190 con 0x7f45d80cc780 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d815a860 con 0x7f45d80c4490 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47250/0 (socket says 172.21.15.98:47250) 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 -- 172.21.15.98:0/4291157052 learned_addr learned my addr 172.21.15.98:0/4291157052 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 -- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45d80c4490 msgr2=0x7f45d80c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45d80c4490 0x7f45d80c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d6d9d700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d80cc780 0x7f45d80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 -- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d80cc780 msgr2=0x7f45d80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d80cc780 0x7f45d80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 -- 172.21.15.98:0/4291157052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009420 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45d659c700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45c800cb60 tx=0x7f45c80017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8005d80 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800a5a0 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800a960 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d815ab40 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.974+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d815a510 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.975+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8064ad0 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.976+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45c800acd0 con 0x7f45d80b0860 2023-10-06T01:40:21.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.976+0000 7f45c67fc700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 0x7f45b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:21.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.976+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f45c80843f0 con 0x7f45d80b0860 2023-10-06T01:40:21.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.976+0000 7f45d5d9b700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 0x7f45b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:21.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.976+0000 7f45d5d9b700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 0x7f45b80693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45d81527b0 tx=0x7f45cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:21.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:21.980+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45c8051290 con 0x7f45d80b0860 2023-10-06T01:40:22.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.164+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45d804bb80 con 0x7f45d80b0860 2023-10-06T01:40:22.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.164+0000 7f45c67fc700 1 -- 172.21.15.98:0/4291157052 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45c80548b0 con 0x7f45d80b0860 2023-10-06T01:40:22.166 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:22.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 msgr2=0x7f45b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 0x7f45b80693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45d81527b0 tx=0x7f45cc017000 comp rx=0 tx=0).stop 2023-10-06T01:40:22.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 msgr2=0x7f45d80c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45c800cb60 tx=0x7f45c80017b0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 shutdown_connections 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f45b8066f30 0x7f45b80693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45d80cc780 0x7f45d80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45d80b0860 0x7f45d80c7980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 --2- 172.21.15.98:0/4291157052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45d80c4490 0x7f45d80c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.166+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 >> 172.21.15.98:0/4291157052 conn(0x7f45d8088000 msgr2=0x7f45d814ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.167+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 shutdown_connections 2023-10-06T01:40:22.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.167+0000 7f45dc937700 1 -- 172.21.15.98:0/4291157052 wait complete. 2023-10-06T01:40:22.179 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:22.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.296+0000 7f92a1ebe700 1 Processor -- start 2023-10-06T01:40:22.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.296+0000 7f92a1ebe700 1 -- start start 2023-10-06T01:40:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f92a1ebe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f929c159340 0x7f929c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f92a1ebe700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c0cf820 con 0x7f929c0cbc40 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f92a1ebe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c0cfb00 con 0x7f929c159710 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f92a1ebe700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c0cfde0 con 0x7f929c159340 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f929b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f929c159340 0x7f929c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f929bfff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f929c0cbc40 0x7f929c0c5660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56292/0 (socket says 172.21.15.98:56292) 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.297+0000 7f929bfff700 1 -- 172.21.15.98:0/211957356 learned_addr learned my addr 172.21.15.98:0/211957356 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929b7fe700 1 -- 172.21.15.98:0/211957356 >> v1:172.21.15.184:6789/0 conn(0x7f929c159710 legacy=0x7f929c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929b7fe700 1 -- 172.21.15.98:0/211957356 >> v1:172.21.15.98:6789/0 conn(0x7f929c0cbc40 legacy=0x7f929c0c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929a7fc700 1 -- 172.21.15.98:0/211957356 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369084085 0 0) 0x7f929c0cf820 con 0x7f929c0cbc40 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929b7fe700 1 -- 172.21.15.98:0/211957356 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0d00c0 con 0x7f929c159340 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929b7fe700 1 --2- 172.21.15.98:0/211957356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c159340 0x7f929c0af4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f928c0098d0 tx=0x7f928c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867cdadfc419f899 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929a7fc700 1 -- 172.21.15.98:0/211957356 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c009e70 con 0x7f929c159340 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929a7fc700 1 -- 172.21.15.98:0/211957356 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c005c10 con 0x7f929c159340 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.298+0000 7f929a7fc700 1 -- 172.21.15.98:0/211957356 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00b650 con 0x7f929c159340 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 -- 172.21.15.98:0/211957356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c159340 msgr2=0x7f929c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/211957356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c159340 0x7f929c0af4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f928c0098d0 tx=0x7f928c009120 comp rx=0 tx=0).stop 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 -- 172.21.15.98:0/211957356 shutdown_connections 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/211957356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c159340 0x7f929c0af4d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 -- 172.21.15.98:0/211957356 >> 172.21.15.98:0/211957356 conn(0x7f929c099720 msgr2=0x7f929c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 -- 172.21.15.98:0/211957356 shutdown_connections 2023-10-06T01:40:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.299+0000 7f92a1ebe700 1 -- 172.21.15.98:0/211957356 wait complete. 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.300+0000 7f92a1ebe700 1 Processor -- start 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.300+0000 7f92a1ebe700 1 -- start start 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 0x7f929c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 0x7f929c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0c8250 con 0x7f929c0cbc40 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c156370 con 0x7f929c159340 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c16d4e0 con 0x7f929c159710 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47266/0 (socket says 172.21.15.98:47266) 2023-10-06T01:40:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 -- 172.21.15.98:0/733304427 learned_addr learned my addr 172.21.15.98:0/733304427 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929bfff700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 0x7f929c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929affd700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 0x7f929c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 -- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 msgr2=0x7f929c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 0x7f929c0ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 -- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 msgr2=0x7f929c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 0x7f929c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 -- 172.21.15.98:0/733304427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0093c0 con 0x7f929c0cbc40 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929affd700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 0x7f929c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929bfff700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 0x7f929c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f929b7fe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f928c009160 tx=0x7f928c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c006460 con 0x7f929c0cbc40 2023-10-06T01:40:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c0065c0 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c16d7c0 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c01a420 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.301+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c16dd80 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.302+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c064ad0 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.303+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f928c01a790 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.303+0000 7f92837fe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 0x7f9288069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.303+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f928c0861f0 con 0x7f929c0cbc40 2023-10-06T01:40:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.303+0000 7f929affd700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 0x7f9288069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.304+0000 7f929affd700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 0x7f9288069460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f929c0c7840 tx=0x7f9290009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.307+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f928c052f70 con 0x7f929c0cbc40 2023-10-06T01:40:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.489+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f929c0afa10 con 0x7f929c0cbc40 2023-10-06T01:40:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.489+0000 7f92837fe700 1 -- 172.21.15.98:0/733304427 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f928c009220 con 0x7f929c0cbc40 2023-10-06T01:40:22.490 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.491+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 msgr2=0x7f9288069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.491+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 0x7f9288069460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f929c0c7840 tx=0x7f9290009040 comp rx=0 tx=0).stop 2023-10-06T01:40:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.491+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 msgr2=0x7f929c155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.491+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f928c009160 tx=0x7f928c016000 comp rx=0 tx=0).stop 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 shutdown_connections 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9288066fb0 0x7f9288069460 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f929c159710 0x7f929c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c0cbc40 0x7f929c155e30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 --2- 172.21.15.98:0/733304427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f929c159340 0x7f929c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 >> 172.21.15.98:0/733304427 conn(0x7f929c099720 msgr2=0x7f929c0ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 shutdown_connections 2023-10-06T01:40:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.492+0000 7f92a1ebe700 1 -- 172.21.15.98:0/733304427 wait complete. 2023-10-06T01:40:22.504 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.620+0000 7f711f59e700 1 Processor -- start 2023-10-06T01:40:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.620+0000 7f711f59e700 1 -- start start 2023-10-06T01:40:22.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711f59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71200b2660 0x7f71200cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711f59e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71200cc9b0 con 0x7f71200cf1f0 2023-10-06T01:40:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711f59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71200cbda0 con 0x7f71200b2a30 2023-10-06T01:40:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711f59e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200cc080 con 0x7f71200b2660 2023-10-06T01:40:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711e59c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71200b2660 0x7f71200cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711ed9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f71200cf1f0 0x7f71200c5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41768/0 (socket says 172.21.15.98:41768) 2023-10-06T01:40:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711ed9d700 1 -- 172.21.15.98:0/2067204673 learned_addr learned my addr 172.21.15.98:0/2067204673 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711e59c700 1 -- 172.21.15.98:0/2067204673 >> v1:172.21.15.184:6789/0 conn(0x7f71200b2a30 legacy=0x7f71200ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711e59c700 1 -- 172.21.15.98:0/2067204673 >> v1:172.21.15.98:6789/0 conn(0x7f71200cf1f0 legacy=0x7f71200c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711e59c700 1 -- 172.21.15.98:0/2067204673 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200cc360 con 0x7f71200b2660 2023-10-06T01:40:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711dd9b700 1 -- 172.21.15.98:0/2067204673 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985301224 0 0) 0x7f71200cc9b0 con 0x7f71200cf1f0 2023-10-06T01:40:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711e59c700 1 --2- 172.21.15.98:0/2067204673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2660 0x7f71200cccf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71100098d0 tx=0x7f7110009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f285a7aa7f77a2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.621+0000 7f711dd9b700 1 -- 172.21.15.98:0/2067204673 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110009e70 con 0x7f71200b2660 2023-10-06T01:40:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711dd9b700 1 -- 172.21.15.98:0/2067204673 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7110005c10 con 0x7f71200b2660 2023-10-06T01:40:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711dd9b700 1 -- 172.21.15.98:0/2067204673 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000b610 con 0x7f71200b2660 2023-10-06T01:40:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 -- 172.21.15.98:0/2067204673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2660 msgr2=0x7f71200cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 --2- 172.21.15.98:0/2067204673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2660 0x7f71200cccf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71100098d0 tx=0x7f7110009120 comp rx=0 tx=0).stop 2023-10-06T01:40:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 -- 172.21.15.98:0/2067204673 shutdown_connections 2023-10-06T01:40:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 --2- 172.21.15.98:0/2067204673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2660 0x7f71200cccf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 -- 172.21.15.98:0/2067204673 >> 172.21.15.98:0/2067204673 conn(0x7f7120099560 msgr2=0x7f712006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 -- 172.21.15.98:0/2067204673 shutdown_connections 2023-10-06T01:40:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.622+0000 7f711f59e700 1 -- 172.21.15.98:0/2067204673 wait complete. 2023-10-06T01:40:22.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.623+0000 7f711f59e700 1 Processor -- start 2023-10-06T01:40:22.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.623+0000 7f711f59e700 1 -- start start 2023-10-06T01:40:22.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71200b2660 0x7f712016cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71200cf1f0 0x7f712014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71201530d0 con 0x7f71200b2a30 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120152da0 con 0x7f71200b2660 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712016d600 con 0x7f71200cf1f0 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53884/0 (socket says 172.21.15.98:53884) 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 -- 172.21.15.98:0/969828037 learned_addr learned my addr 172.21.15.98:0/969828037 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711e59c700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71200b2660 0x7f712016cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f711ed9d700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71200cf1f0 0x7f712014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 -- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71200cf1f0 msgr2=0x7f712014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71200cf1f0 0x7f712014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 -- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71200b2660 msgr2=0x7f712016cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71200b2660 0x7f712016cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.624+0000 7f7115d9b700 1 -- 172.21.15.98:0/969828037 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100093c0 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f7115d9b700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f710800c590 tx=0x7f7108007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108011590 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71080116f0 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712016d8e0 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108011ab0 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.625+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712016d240 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.626+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712005f270 con 0x7f71200b2a30 2023-10-06T01:40:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.626+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f710801c460 con 0x7f71200b2a30 2023-10-06T01:40:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.627+0000 7f7116ffd700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.627+0000 7f711e59c700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.627+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7108084590 con 0x7f71200b2a30 2023-10-06T01:40:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.627+0000 7f711e59c700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 0x7f71000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7110009390 tx=0x7f7110015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.631+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71080513b0 con 0x7f71200b2a30 2023-10-06T01:40:22.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.811+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f71200cc640 con 0x7f71200b2a30 2023-10-06T01:40:22.812 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.812+0000 7f7116ffd700 1 -- 172.21.15.98:0/969828037 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f71080549d0 con 0x7f71200b2a30 2023-10-06T01:40:22.812 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:22.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 msgr2=0x7f71000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 0x7f71000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7110009390 tx=0x7f7110015000 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 msgr2=0x7f712014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f710800c590 tx=0x7f7108007f30 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 shutdown_connections 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71200cf1f0 0x7f712014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71200b2660 0x7f712016cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.813+0000 7f711f59e700 1 --2- 172.21.15.98:0/969828037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71200b2a30 0x7f712014eea0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.814+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 >> 172.21.15.98:0/969828037 conn(0x7f7120099560 msgr2=0x7f7120068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.814+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 shutdown_connections 2023-10-06T01:40:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.814+0000 7f711f59e700 1 -- 172.21.15.98:0/969828037 wait complete. 2023-10-06T01:40:22.826 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:22.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.940+0000 7f4dfaf06700 1 Processor -- start 2023-10-06T01:40:22.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.940+0000 7f4dfaf06700 1 -- start start 2023-10-06T01:40:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4dfaf06700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4df40aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4dfaf06700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4df40b07a0 0x7f4df40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4dfaf06700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40b0ca0 con 0x7f4df41595a0 2023-10-06T01:40:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4dfaf06700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40b0f80 con 0x7f4df40b07a0 2023-10-06T01:40:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4dfaf06700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40b1260 con 0x7f4df40b49e0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4df8ca2700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4df41595a0 0x7f4df40aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41772/0 (socket says 172.21.15.98:41772) 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.941+0000 7f4df8ca2700 1 -- 172.21.15.98:0/242713522 learned_addr learned my addr 172.21.15.98:0/242713522 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df3fff700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4df40aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 --2- 172.21.15.98:0/242713522 >> v2:172.21.15.184:3300/0 conn(0x7f4df40b07a0 0x7f4df40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89430763 0 0) 0x7f4df40b0ca0 con 0x7f4df41595a0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de0003570 con 0x7f4df41595a0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4254973138 0 0) 0x7f4de0003570 con 0x7f4df41595a0 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 msgr2=0x7f4df40aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4df40aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 >> v2:172.21.15.184:3300/0 conn(0x7f4df40b07a0 msgr2=0x7f4df40b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 --2- 172.21.15.98:0/242713522 >> v2:172.21.15.184:3300/0 conn(0x7f4df40b07a0 0x7f4df40b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40b1540 con 0x7f4df41595a0 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f4de40015f0 con 0x7f4df41595a0 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df3fff700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4df40aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 >> v1:172.21.15.98:6789/0 conn(0x7f4df41595a0 legacy=0x7f4df40aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4de0003a60 0x7f4de0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de0004d50 0x7f4de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40b0f80 con 0x7f4de0004d50 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40b1260 con 0x7f4de0003a60 2023-10-06T01:40:22.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de000ba60 con 0x7f4df40b49e0 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de0004d50 0x7f4de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df8ca2700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4de0003a60 0x7f4de0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df3fff700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 -- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 msgr2=0x7f4de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4df40b49e0 0x7f4de0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 -- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4de0003a60 msgr2=0x7f4de0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4de0003a60 0x7f4de0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 -- 172.21.15.98:0/242713522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009a20 con 0x7f4de0004d50 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.942+0000 7f4df94a3700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de0004d50 0x7f4de00051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009c00 tx=0x7f4dec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e568d78ef84488 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00e6b0 con 0x7f4de0004d50 2023-10-06T01:40:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00e810 con 0x7f4de0004d50 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4df37fe700 1 -- 172.21.15.98:0/242713522 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00ebd0 con 0x7f4de0004d50 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 -- 172.21.15.98:0/242713522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de0004d50 msgr2=0x7f4de00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/242713522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de0004d50 0x7f4de00051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009c00 tx=0x7f4dec0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4df94a3700 1 -- 172.21.15.98:0/242713522 reap_dead start 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 -- 172.21.15.98:0/242713522 shutdown_connections 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 -- 172.21.15.98:0/242713522 >> 172.21.15.98:0/242713522 conn(0x7f4df4099830 msgr2=0x7f4df406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 -- 172.21.15.98:0/242713522 shutdown_connections 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.943+0000 7f4dfaf06700 1 -- 172.21.15.98:0/242713522 wait complete. 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.944+0000 7f4dfaf06700 1 Processor -- start 2023-10-06T01:40:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.944+0000 7f4dfaf06700 1 -- start start 2023-10-06T01:40:22.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4de0004d50 0x7f4df415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4df40af0b0 0x7f4df409bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df409a090 con 0x7f4df409dd50 2023-10-06T01:40:22.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df409a930 con 0x7f4df40af0b0 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4dfaf06700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df416a960 con 0x7f4de0004d50 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53902/0 (socket says 172.21.15.98:53902) 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4df3fff700 1 -- 172.21.15.98:0/2606776015 learned_addr learned my addr 172.21.15.98:0/2606776015 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4df94a3700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4df40af0b0 0x7f4df409bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.945+0000 7f4df8ca2700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4de0004d50 0x7f4df415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 -- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4de0004d50 msgr2=0x7f4df415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4de0004d50 0x7f4df415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 -- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4df40af0b0 msgr2=0x7f4df409bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4df40af0b0 0x7f4df409bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:22.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 -- 172.21.15.98:0/2606776015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df409a2e0 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df3fff700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00f260 tx=0x7f4ddc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00e040 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc00f040 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df409a5c0 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc015430 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.946+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df416a530 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.947+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df409aac0 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.947+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ddc0157e0 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.947+0000 7f4df0ff9700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 0x7f4dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.948+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4ddc0856b0 con 0x7f4df409dd50 2023-10-06T01:40:22.954 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.948+0000 7f4df8ca2700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 0x7f4dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:22.955 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.948+0000 7f4df8ca2700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 0x7f4dd40693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4de40011d0 tx=0x7f4de4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:22.955 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:22.952+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ddc0524d0 con 0x7f4df409dd50 2023-10-06T01:40:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.135+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4df4064ad0 con 0x7f4df409dd50 2023-10-06T01:40:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.136+0000 7f4df0ff9700 1 -- 172.21.15.98:0/2606776015 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f4ddc055af0 con 0x7f4df409dd50 2023-10-06T01:40:23.136 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:23.137 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:23.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 msgr2=0x7f4dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 0x7f4dd40693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4de40011d0 tx=0x7f4de4009040 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 msgr2=0x7f4df409adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00f260 tx=0x7f4ddc00b040 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 shutdown_connections 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4dd4066f30 0x7f4dd40693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4df40af0b0 0x7f4df409bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4de0004d50 0x7f4df415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 --2- 172.21.15.98:0/2606776015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4df409dd50 0x7f4df409adc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.137+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 >> 172.21.15.98:0/2606776015 conn(0x7f4df4099830 msgr2=0x7f4df4158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.138+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 shutdown_connections 2023-10-06T01:40:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.138+0000 7f4dfaf06700 1 -- 172.21.15.98:0/2606776015 wait complete. 2023-10-06T01:40:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:23.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.267+0000 7feac11b1700 1 Processor -- start 2023-10-06T01:40:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.267+0000 7feac11b1700 1 -- start start 2023-10-06T01:40:23.270 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 0x7feabc0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.270 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 0x7feabc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feabc0ce820 0x7feabc0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0cf610 con 0x7feabc0ce820 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0cf8f0 con 0x7feabc1598e0 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feac11b1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0cfbd0 con 0x7feabc159510 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feabc0ce820 0x7feabc0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feabc0ce820 0x7feabc0cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53904/0 (socket says 172.21.15.98:53904) 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 -- 172.21.15.98:0/72832083 learned_addr learned my addr 172.21.15.98:0/72832083 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabad9d700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 0x7feabc0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feab259c700 1 --2- 172.21.15.98:0/72832083 >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 0x7feabc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 -- 172.21.15.98:0/72832083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 msgr2=0x7feabc0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:23.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 0x7feabc0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 -- 172.21.15.98:0/72832083 >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 msgr2=0x7feabc0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.268+0000 7feabb59e700 1 --2- 172.21.15.98:0/72832083 >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 0x7feabc0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feabb59e700 1 -- 172.21.15.98:0/72832083 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0cfeb0 con 0x7feabc0ce820 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feabad9d700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 0x7feabc0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feab259c700 1 --2- 172.21.15.98:0/72832083 >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 0x7feabc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feabb59e700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc0ce820 0x7feabc0cec40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feabc089360 tx=0x7feab4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33916f2d53ef23d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feaba59c700 1 -- 172.21.15.98:0/72832083 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400e070 con 0x7feabc0ce820 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feaba59c700 1 -- 172.21.15.98:0/72832083 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400ba50 con 0x7feabc0ce820 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feaba59c700 1 -- 172.21.15.98:0/72832083 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40092e0 con 0x7feabc0ce820 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 -- 172.21.15.98:0/72832083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc0ce820 msgr2=0x7feabc0cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc0ce820 0x7feabc0cec40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feabc089360 tx=0x7feab4009840 comp rx=0 tx=0).stop 2023-10-06T01:40:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 -- 172.21.15.98:0/72832083 shutdown_connections 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc0ce820 0x7feabc0cec40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 --2- 172.21.15.98:0/72832083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc159510 0x7feabc0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 --2- 172.21.15.98:0/72832083 >> v2:172.21.15.184:3300/0 conn(0x7feabc1598e0 0x7feabc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 -- 172.21.15.98:0/72832083 >> 172.21.15.98:0/72832083 conn(0x7feabc089e90 msgr2=0x7feabc0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 -- 172.21.15.98:0/72832083 shutdown_connections 2023-10-06T01:40:23.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.269+0000 7feac11b1700 1 -- 172.21.15.98:0/72832083 wait complete. 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 Processor -- start 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 -- start start 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc0ce820 0x7feabc14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feabc159510 0x7feabc0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc16cfb0 con 0x7feabc1598e0 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b4100 con 0x7feabc159510 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feac11b1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b43e0 con 0x7feabc0ce820 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feabb59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feabb59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53908/0 (socket says 172.21.15.98:53908) 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.271+0000 7feabb59e700 1 -- 172.21.15.98:0/3994273804 learned_addr learned my addr 172.21.15.98:0/3994273804 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabad9d700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc0ce820 0x7feabc14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feab259c700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feabc159510 0x7feabc0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 -- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc0ce820 msgr2=0x7feabc14a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc0ce820 0x7feabc14a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 -- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feabc159510 msgr2=0x7feabc0c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feabc159510 0x7feabc0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 -- 172.21.15.98:0/3994273804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009ae0 con 0x7feabc1598e0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feabb59e700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feabc067c60 tx=0x7feab4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40032f0 con 0x7feabc1598e0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4008e80 con 0x7feabc1598e0 2023-10-06T01:40:23.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc0b46c0 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4025690 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.272+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc0b4c80 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.273+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc064ad0 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.273+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feab4003040 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.274+0000 7feab37fe700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 0x7fea9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.274+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7feab40867f0 con 0x7feabc1598e0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.274+0000 7feabad9d700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 0x7fea9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.274+0000 7feabad9d700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 0x7fea9c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feaac0089b0 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.278+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feab40536c0 con 0x7feabc1598e0 2023-10-06T01:40:23.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.460+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feabc0cf180 con 0x7feabc1598e0 2023-10-06T01:40:23.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.460+0000 7feab37fe700 1 -- 172.21.15.98:0/3994273804 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7feab4056ce0 con 0x7feabc1598e0 2023-10-06T01:40:23.461 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 msgr2=0x7fea9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 0x7fea9c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feaac0089b0 tx=0x7feaac008040 comp rx=0 tx=0).stop 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 msgr2=0x7feabc0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feabc067c60 tx=0x7feab4018000 comp rx=0 tx=0).stop 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 shutdown_connections 2023-10-06T01:40:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fea9c066f30 0x7fea9c0693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feabc1598e0 0x7feabc0c6840 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feabc0ce820 0x7feabc14a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 --2- 172.21.15.98:0/3994273804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feabc159510 0x7feabc0c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 >> 172.21.15.98:0/3994273804 conn(0x7feabc089e90 msgr2=0x7feabc0cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 shutdown_connections 2023-10-06T01:40:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.462+0000 7feac11b1700 1 -- 172.21.15.98:0/3994273804 wait complete. 2023-10-06T01:40:23.475 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:23.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.591+0000 7f9331c52700 1 Processor -- start 2023-10-06T01:40:23.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.591+0000 7f9331c52700 1 -- start start 2023-10-06T01:40:23.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f9331c52700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f932c1598e0 0x7f932c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f9331c52700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f932c0cbcb0 0x7f932c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f9331c52700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0cf610 con 0x7f932c159510 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f9331c52700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0cf8f0 con 0x7f932c1598e0 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f9331c52700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0cfbd0 con 0x7f932c0cbcb0 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f932affd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f932c1598e0 0x7f932c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f932affd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f932c1598e0 0x7f932c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53918/0 (socket says 172.21.15.98:53918) 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f932affd700 1 -- 172.21.15.98:0/3826539023 learned_addr learned my addr 172.21.15.98:0/3826539023 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.592+0000 7f932bfff700 1 --2- 172.21.15.98:0/3826539023 >> v2:172.21.15.184:3300/0 conn(0x7f932c0cbcb0 0x7f932c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932affd700 1 -- 172.21.15.98:0/3826539023 >> v2:172.21.15.184:3300/0 conn(0x7f932c0cbcb0 msgr2=0x7f932c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932affd700 1 --2- 172.21.15.98:0/3826539023 >> v2:172.21.15.184:3300/0 conn(0x7f932c0cbcb0 0x7f932c0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932affd700 1 -- 172.21.15.98:0/3826539023 >> v1:172.21.15.184:6789/0 conn(0x7f932c159510 legacy=0x7f932c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932affd700 1 -- 172.21.15.98:0/3826539023 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0cfeb0 con 0x7f932c1598e0 2023-10-06T01:40:23.595 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932affd700 1 --2- 172.21.15.98:0/3826539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c1598e0 0x7f932c0ce820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f932c06f990 tx=0x7f9314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54388e394924350 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932a7fc700 1 -- 172.21.15.98:0/3826539023 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314009e70 con 0x7f932c1598e0 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932a7fc700 1 -- 172.21.15.98:0/3826539023 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400b860 con 0x7f932c1598e0 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f932a7fc700 1 -- 172.21.15.98:0/3826539023 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bc00 con 0x7f932c1598e0 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 -- 172.21.15.98:0/3826539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c1598e0 msgr2=0x7f932c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 --2- 172.21.15.98:0/3826539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c1598e0 0x7f932c0ce820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f932c06f990 tx=0x7f9314009250 comp rx=0 tx=0).stop 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 -- 172.21.15.98:0/3826539023 shutdown_connections 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 --2- 172.21.15.98:0/3826539023 >> v2:172.21.15.184:3300/0 conn(0x7f932c0cbcb0 0x7f932c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 --2- 172.21.15.98:0/3826539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c1598e0 0x7f932c0ce820 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.593+0000 7f9331c52700 1 -- 172.21.15.98:0/3826539023 >> 172.21.15.98:0/3826539023 conn(0x7f932c089e90 msgr2=0x7f932c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.594+0000 7f9331c52700 1 -- 172.21.15.98:0/3826539023 shutdown_connections 2023-10-06T01:40:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.594+0000 7f9331c52700 1 -- 172.21.15.98:0/3826539023 wait complete. 2023-10-06T01:40:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.595+0000 7f9331c52700 1 Processor -- start 2023-10-06T01:40:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.595+0000 7f9331c52700 1 -- start start 2023-10-06T01:40:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.595+0000 7f9331c52700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0cbcb0 0x7f932c169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.595+0000 7f9331c52700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.595+0000 7f9331c52700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c1598e0 0x7f932c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f9331c52700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c159ce0 con 0x7f932c159510 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f9331c52700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c14e5f0 con 0x7f932c0cbcb0 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f9331c52700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c159ef0 con 0x7f932c1598e0 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53930/0 (socket says 172.21.15.98:53930) 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 -- 172.21.15.98:0/3596669468 learned_addr learned my addr 172.21.15.98:0/3596669468 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932b7fe700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0cbcb0 0x7f932c169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932bfff700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c1598e0 0x7f932c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.600 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 -- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c1598e0 msgr2=0x7f932c155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c1598e0 0x7f932c155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 -- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0cbcb0 msgr2=0x7f932c169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0cbcb0 0x7f932c169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 -- 172.21.15.98:0/3596669468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140094f0 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f932affd700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f932c067c60 tx=0x7f9314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bbe0 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314011420 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c15a1d0 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140117c0 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.596+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c15a790 con 0x7f932c159510 2023-10-06T01:40:23.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.597+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c064ad0 con 0x7f932c159510 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.598+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9314011bf0 con 0x7f932c159510 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.598+0000 7f93237fe700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.598+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9314085dc0 con 0x7f932c159510 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.598+0000 7f932b7fe700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.598+0000 7f932b7fe700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 0x7f930c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c0016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.602+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9314052be0 con 0x7f932c159510 2023-10-06T01:40:23.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.783+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f932c0cf2a0 con 0x7f932c159510 2023-10-06T01:40:23.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.784+0000 7f93237fe700 1 -- 172.21.15.98:0/3596669468 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9314056200 con 0x7f932c159510 2023-10-06T01:40:23.784 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 msgr2=0x7f930c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 0x7f930c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c0016f0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 msgr2=0x7f932c14f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f932c067c60 tx=0x7f9314017000 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 shutdown_connections 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f932c1598e0 0x7f932c155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c0cbcb0 0x7f932c169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 --2- 172.21.15.98:0/3596669468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c159510 0x7f932c14f170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 >> 172.21.15.98:0/3596669468 conn(0x7f932c089e90 msgr2=0x7f932c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 shutdown_connections 2023-10-06T01:40:23.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.786+0000 7f9331c52700 1 -- 172.21.15.98:0/3596669468 wait complete. 2023-10-06T01:40:23.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:23.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.915+0000 7f7afe0bc700 1 Processor -- start 2023-10-06T01:40:23.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.915+0000 7f7afe0bc700 1 -- start start 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afe0bc700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7af80b2660 0x7f7af80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afe0bc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7af80b2a30 0x7f7af80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afe0bc700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af80cc9b0 con 0x7f7af80ceae0 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afe0bc700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80cbda0 con 0x7f7af80b2a30 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afe0bc700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80cc080 con 0x7f7af80b2660 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7af80b2a30 0x7f7af80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7af80b2a30 0x7f7af80cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53940/0 (socket says 172.21.15.98:53940) 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1901654890 learned_addr learned my addr 172.21.15.98:0/1901654890 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afd0ba700 1 --2- 172.21.15.98:0/1901654890 >> v2:172.21.15.184:3300/0 conn(0x7f7af80b2660 0x7f7af80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.918 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1901654890 >> v2:172.21.15.184:3300/0 conn(0x7f7af80b2660 msgr2=0x7f7af80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 --2- 172.21.15.98:0/1901654890 >> v2:172.21.15.184:3300/0 conn(0x7f7af80b2660 0x7f7af80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1901654890 >> v1:172.21.15.184:6789/0 conn(0x7f7af80ceae0 legacy=0x7f7af80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1901654890 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80cc360 con 0x7f7af80b2a30 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7afc8b9700 1 --2- 172.21.15.98:0/1901654890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af80cd230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7af8089500 tx=0x7f7ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad1ef70ae4aeac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.916+0000 7f7aeffff700 1 -- 172.21.15.98:0/1901654890 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800b820 con 0x7f7af80b2a30 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7aeffff700 1 -- 172.21.15.98:0/1901654890 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae800b980 con 0x7f7af80b2a30 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7aeffff700 1 -- 172.21.15.98:0/1901654890 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800bd20 con 0x7f7af80b2a30 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1901654890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 msgr2=0x7f7af80cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1901654890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af80cd230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7af8089500 tx=0x7f7ae8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1901654890 shutdown_connections 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1901654890 >> v2:172.21.15.184:3300/0 conn(0x7f7af80b2660 0x7f7af80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.919 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1901654890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af80cd230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.920 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1901654890 >> 172.21.15.98:0/1901654890 conn(0x7f7af8099560 msgr2=0x7f7af806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:23.920 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1901654890 shutdown_connections 2023-10-06T01:40:23.920 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.917+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1901654890 wait complete. 2023-10-06T01:40:23.922 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.918+0000 7f7afe0bc700 1 Processor -- start 2023-10-06T01:40:23.922 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.918+0000 7f7afe0bc700 1 -- start start 2023-10-06T01:40:23.922 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7af80b2660 0x7f7af813f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.922 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.922 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af80ceae0 0x7f7af816cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af813f950 con 0x7f7af80b2a30 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80c7b00 con 0x7f7af80ceae0 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afe0bc700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af8155920 con 0x7f7af80b2660 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53956/0 (socket says 172.21.15.98:53956) 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1413776181 learned_addr learned my addr 172.21.15.98:0/1413776181 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afd8bb700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af80ceae0 0x7f7af816cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afd0ba700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7af80b2660 0x7f7af813f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7af80b2660 msgr2=0x7f7af813f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7af80b2660 0x7f7af813f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.923 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af80ceae0 msgr2=0x7f7af816cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af80ceae0 0x7f7af816cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.919+0000 7f7afc8b9700 1 -- 172.21.15.98:0/1413776181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80094f0 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7afc8b9700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009290 tx=0x7f7ae8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8011e20 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae8020070 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80c7540 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801b6a0 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.920+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af81555b0 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.921+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af8155420 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.921+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ae8022030 con 0x7f7af80b2a30 2023-10-06T01:40:23.924 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.921+0000 7f7aed7fa700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:23.925 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.921+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7ae80868d0 con 0x7f7af80b2a30 2023-10-06T01:40:23.925 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.922+0000 7f7afd0ba700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:23.925 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.922+0000 7f7afd0ba700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7af4006520 tx=0x7f7af4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:23.926 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:23.926+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ae80536f0 con 0x7f7af80b2a30 2023-10-06T01:40:24.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.105+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7af8155420 con 0x7f7af80b2a30 2023-10-06T01:40:24.106 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.106+0000 7f7aed7fa700 1 -- 172.21.15.98:0/1413776181 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ae8056d10 con 0x7f7af80b2a30 2023-10-06T01:40:24.107 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 msgr2=0x7f7adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7af4006520 tx=0x7f7af4008040 comp rx=0 tx=0).stop 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 msgr2=0x7f7af8155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009290 tx=0x7f7ae8016000 comp rx=0 tx=0).stop 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 shutdown_connections 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7af80ceae0 0x7f7af816cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7af80b2660 0x7f7af813f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 --2- 172.21.15.98:0/1413776181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7af80b2a30 0x7f7af8155db0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 >> 172.21.15.98:0/1413776181 conn(0x7f7af8099560 msgr2=0x7f7af80cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 shutdown_connections 2023-10-06T01:40:24.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.108+0000 7f7afe0bc700 1 -- 172.21.15.98:0/1413776181 wait complete. 2023-10-06T01:40:24.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:24.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.239+0000 7fb19179f700 1 Processor -- start 2023-10-06T01:40:24.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.239+0000 7fb19179f700 1 -- start start 2023-10-06T01:40:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb19179f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 0x7fb18c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb19179f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb18c0b2a30 0x7fb18c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb19179f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0cc9b0 con 0x7fb18c0ceae0 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb19179f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c0cbda0 con 0x7fb18c0b2a30 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb19179f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0cc080 con 0x7fb18c0b2660 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb18c0b2a30 0x7fb18c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb18c0b2a30 0x7fb18c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53968/0 (socket says 172.21.15.98:53968) 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 -- 172.21.15.98:0/1970646730 learned_addr learned my addr 172.21.15.98:0/1970646730 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18bfff700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 0x7fb18c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 -- 172.21.15.98:0/1970646730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 msgr2=0x7fb18c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 0x7fb18c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 -- 172.21.15.98:0/1970646730 >> v1:172.21.15.184:6789/0 conn(0x7fb18c0ceae0 legacy=0x7fb18c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18b7fe700 1 -- 172.21.15.98:0/1970646730 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0cc360 con 0x7fb18c0b2a30 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.240+0000 7fb18bfff700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 0x7fb18c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb18b7fe700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c0cd230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1800099b0 tx=0x7fb180009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a56a256b21f37e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb18affd700 1 -- 172.21.15.98:0/1970646730 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18000b820 con 0x7fb18c0b2a30 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb18affd700 1 -- 172.21.15.98:0/1970646730 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18000b980 con 0x7fb18c0b2a30 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb18affd700 1 -- 172.21.15.98:0/1970646730 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18000bd20 con 0x7fb18c0b2a30 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 -- 172.21.15.98:0/1970646730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 msgr2=0x7fb18c0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c0cd230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1800099b0 tx=0x7fb180009250 comp rx=0 tx=0).stop 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 -- 172.21.15.98:0/1970646730 shutdown_connections 2023-10-06T01:40:24.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0b2660 0x7fb18c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 --2- 172.21.15.98:0/1970646730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c0cd230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 -- 172.21.15.98:0/1970646730 >> 172.21.15.98:0/1970646730 conn(0x7fb18c099560 msgr2=0x7fb18c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 -- 172.21.15.98:0/1970646730 shutdown_connections 2023-10-06T01:40:24.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.241+0000 7fb19179f700 1 -- 172.21.15.98:0/1970646730 wait complete. 2023-10-06T01:40:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.242+0000 7fb19179f700 1 Processor -- start 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 -- start start 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 0x7fb18c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 0x7fb18c15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c157a40 con 0x7fb18c0b2a30 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0b4af0 con 0x7fb18c0b2660 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb19179f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0b4dd0 con 0x7fb18c0ceae0 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb18b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb18b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53980/0 (socket says 172.21.15.98:53980) 2023-10-06T01:40:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.243+0000 7fb18b7fe700 1 -- 172.21.15.98:0/3861573808 learned_addr learned my addr 172.21.15.98:0/3861573808 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18bfff700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 0x7fb18c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb190f9e700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 0x7fb18c15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 -- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 msgr2=0x7fb18c15ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 0x7fb18c15ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 -- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 msgr2=0x7fb18c0b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 0x7fb18c0b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 -- 172.21.15.98:0/3861573808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800094f0 con 0x7fb18c0b2a30 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18bfff700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 0x7fb18c0b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb190f9e700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 0x7fb18c15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb18b7fe700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb180009290 tx=0x7fb180016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180011d90 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180020070 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0b50b0 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18001b6a0 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.244+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c0b5670 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.245+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c0b5860 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.245+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb180022030 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.246+0000 7fb173fff700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 0x7fb1780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.246+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb180086a10 con 0x7fb18c0b2a30 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.246+0000 7fb18bfff700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 0x7fb1780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.246+0000 7fb18bfff700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 0x7fb1780693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ab10 tx=0x7fb17c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.250+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb180053830 con 0x7fb18c0b2a30 2023-10-06T01:40:24.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.435+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb18c0b5860 con 0x7fb18c0b2a30 2023-10-06T01:40:24.436 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.435+0000 7fb173fff700 1 -- 172.21.15.98:0/3861573808 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb180056e50 con 0x7fb18c0b2a30 2023-10-06T01:40:24.436 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 msgr2=0x7fb1780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 0x7fb1780693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ab10 tx=0x7fb17c008040 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 msgr2=0x7fb18c15cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb180009290 tx=0x7fb180016000 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 shutdown_connections 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb178066f30 0x7fb1780693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18c0ceae0 0x7fb18c15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18c0b2660 0x7fb18c0b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 --2- 172.21.15.98:0/3861573808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb18c0b2a30 0x7fb18c15cfa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 >> 172.21.15.98:0/3861573808 conn(0x7fb18c099560 msgr2=0x7fb18c157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 shutdown_connections 2023-10-06T01:40:24.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.438+0000 7fb19179f700 1 -- 172.21.15.98:0/3861573808 wait complete. 2023-10-06T01:40:24.450 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:24.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.566+0000 7fae0d689700 1 Processor -- start 2023-10-06T01:40:24.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 -- start start 2023-10-06T01:40:24.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae08159510 0x7fae080cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae081598e0 0x7fae080cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae080cf370 con 0x7fae080ce820 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cf610 con 0x7fae08159510 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae0d689700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cf8f0 con 0x7fae081598e0 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.567+0000 7fae06ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae08159510 0x7fae080cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae08159510 0x7fae080cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53984/0 (socket says 172.21.15.98:53984) 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 -- 172.21.15.98:0/2658571250 learned_addr learned my addr 172.21.15.98:0/2658571250 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fadfe7fc700 1 --2- 172.21.15.98:0/2658571250 >> v2:172.21.15.184:3300/0 conn(0x7fae081598e0 0x7fae080cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 -- 172.21.15.98:0/2658571250 >> v2:172.21.15.184:3300/0 conn(0x7fae081598e0 msgr2=0x7fae080cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 --2- 172.21.15.98:0/2658571250 >> v2:172.21.15.184:3300/0 conn(0x7fae081598e0 0x7fae080cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 -- 172.21.15.98:0/2658571250 >> v1:172.21.15.184:6789/0 conn(0x7fae080ce820 legacy=0x7fae080cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 -- 172.21.15.98:0/2658571250 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080cfbd0 con 0x7fae08159510 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae06ffd700 1 --2- 172.21.15.98:0/2658571250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae08159510 0x7fae080cb5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fae080c73b0 tx=0x7fadf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1962fafdfd0ce31d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae067fc700 1 -- 172.21.15.98:0/2658571250 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8009e70 con 0x7fae08159510 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae067fc700 1 -- 172.21.15.98:0/2658571250 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf800b860 con 0x7fae08159510 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.568+0000 7fae067fc700 1 -- 172.21.15.98:0/2658571250 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf800bc00 con 0x7fae08159510 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 -- 172.21.15.98:0/2658571250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae08159510 msgr2=0x7fae080cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 --2- 172.21.15.98:0/2658571250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae08159510 0x7fae080cb5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fae080c73b0 tx=0x7fadf8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 -- 172.21.15.98:0/2658571250 shutdown_connections 2023-10-06T01:40:24.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 --2- 172.21.15.98:0/2658571250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae08159510 0x7fae080cb5a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 --2- 172.21.15.98:0/2658571250 >> v2:172.21.15.184:3300/0 conn(0x7fae081598e0 0x7fae080cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 -- 172.21.15.98:0/2658571250 >> 172.21.15.98:0/2658571250 conn(0x7fae08089e90 msgr2=0x7fae080997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 -- 172.21.15.98:0/2658571250 shutdown_connections 2023-10-06T01:40:24.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.569+0000 7fae0d689700 1 -- 172.21.15.98:0/2658571250 wait complete. 2023-10-06T01:40:24.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.570+0000 7fae0d689700 1 Processor -- start 2023-10-06T01:40:24.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.570+0000 7fae0d689700 1 -- start start 2023-10-06T01:40:24.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae08159510 0x7fae0809cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae081598e0 0x7fae0809d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0809e1c0 con 0x7fae080ce820 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0809d640 con 0x7fae08159510 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0809d920 con 0x7fae081598e0 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53992/0 (socket says 172.21.15.98:53992) 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 -- 172.21.15.98:0/3581758491 learned_addr learned my addr 172.21.15.98:0/3581758491 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 -- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae081598e0 msgr2=0x7fae0809d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae081598e0 0x7fae0809d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fadfe7fc700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae08159510 0x7fae0809cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 -- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae08159510 msgr2=0x7fae0809cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae08159510 0x7fae0809cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 -- 172.21.15.98:0/3581758491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80094f0 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae06ffd700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae08067c60 tx=0x7fadf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8011420 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80115a0 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8011960 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0809dc00 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.571+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0814f6c0 con 0x7fae080ce820 2023-10-06T01:40:24.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.572+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08064ad0 con 0x7fae080ce820 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.573+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadf80253f0 con 0x7fae080ce820 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.573+0000 7fadff7fe700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 0x7fade80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.573+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fadf8086340 con 0x7fae080ce820 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.573+0000 7fadfe7fc700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 0x7fade80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.573+0000 7fadfe7fc700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 0x7fade80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006f00 tx=0x7fadf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.577+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadf8053160 con 0x7fae080ce820 2023-10-06T01:40:24.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.757+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fae080cfeb0 con 0x7fae080ce820 2023-10-06T01:40:24.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.757+0000 7fadff7fe700 1 -- 172.21.15.98:0/3581758491 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fadf8056780 con 0x7fae080ce820 2023-10-06T01:40:24.758 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:24.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 msgr2=0x7fade80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 0x7fade80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006f00 tx=0x7fadf0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:24.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 msgr2=0x7fae0814f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae08067c60 tx=0x7fadf8017000 comp rx=0 tx=0).stop 2023-10-06T01:40:24.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 shutdown_connections 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fade8066f30 0x7fade80693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae081598e0 0x7fae0809d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae080ce820 0x7fae0814f180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 --2- 172.21.15.98:0/3581758491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae08159510 0x7fae0809cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 >> 172.21.15.98:0/3581758491 conn(0x7fae08089e90 msgr2=0x7fae080c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 shutdown_connections 2023-10-06T01:40:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.759+0000 7fae0d689700 1 -- 172.21.15.98:0/3581758491 wait complete. 2023-10-06T01:40:24.783 INFO:tasks.cephfs_test_runner:Starting test: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2023-10-06T01:40:24.783 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2023-10-06T01:40:24.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.896+0000 7f57da6a5700 1 Processor -- start 2023-10-06T01:40:24.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.896+0000 7f57da6a5700 1 -- start start 2023-10-06T01:40:24.899 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.897+0000 7f57da6a5700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57d40ad250 0x7f57d40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.897+0000 7f57da6a5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.897+0000 7f57da6a5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40ced30 con 0x7f57d415d980 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.897+0000 7f57da6a5700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d415de20 con 0x7f57d40ad250 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.897+0000 7f57da6a5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40cf010 con 0x7f57d40ad620 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:32912/0 (socket says 172.21.15.98:32912) 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 -- 172.21.15.98:0/3879608899 learned_addr learned my addr 172.21.15.98:0/3879608899 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d3fff700 1 --2- 172.21.15.98:0/3879608899 >> v2:172.21.15.184:3300/0 conn(0x7f57d40ad250 0x7f57d40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d2ffd700 1 -- 172.21.15.98:0/3879608899 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909202093 0 0) 0x7f57d40ced30 con 0x7f57d415d980 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d2ffd700 1 -- 172.21.15.98:0/3879608899 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57bc003570 con 0x7f57d415d980 2023-10-06T01:40:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 -- 172.21.15.98:0/3879608899 >> v2:172.21.15.184:3300/0 conn(0x7f57d40ad250 msgr2=0x7f57d40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 --2- 172.21.15.98:0/3879608899 >> v2:172.21.15.184:3300/0 conn(0x7f57d40ad250 0x7f57d40cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 -- 172.21.15.98:0/3879608899 >> v1:172.21.15.184:6789/0 conn(0x7f57d415d980 legacy=0x7f57d4068700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 -- 172.21.15.98:0/3879608899 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40ccad0 con 0x7f57d40ad620 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d37fe700 1 --2- 172.21.15.98:0/3879608899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57c80099b0 tx=0x7f57c8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c162d2df7aff8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d2ffd700 1 -- 172.21.15.98:0/3879608899 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800b680 con 0x7f57d40ad620 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d2ffd700 1 -- 172.21.15.98:0/3879608899 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800b7e0 con 0x7f57d40ad620 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.898+0000 7f57d2ffd700 1 -- 172.21.15.98:0/3879608899 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800bb60 con 0x7f57d40ad620 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 -- 172.21.15.98:0/3879608899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 msgr2=0x7f57d415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 --2- 172.21.15.98:0/3879608899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57c80099b0 tx=0x7f57c8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 -- 172.21.15.98:0/3879608899 shutdown_connections 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 --2- 172.21.15.98:0/3879608899 >> v2:172.21.15.184:3300/0 conn(0x7f57d40ad250 0x7f57d40cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 --2- 172.21.15.98:0/3879608899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d415d440 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 -- 172.21.15.98:0/3879608899 >> 172.21.15.98:0/3879608899 conn(0x7f57d4067dd0 msgr2=0x7f57d40ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:24.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 -- 172.21.15.98:0/3879608899 shutdown_connections 2023-10-06T01:40:24.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.899+0000 7f57da6a5700 1 -- 172.21.15.98:0/3879608899 wait complete. 2023-10-06T01:40:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.900+0000 7f57da6a5700 1 Processor -- start 2023-10-06T01:40:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.900+0000 7f57da6a5700 1 -- start start 2023-10-06T01:40:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d40ad250 0x7f57d4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d40ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40aff00 con 0x7f57d415d980 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d415a1a0 con 0x7f57d40ad250 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57da6a5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40af2f0 con 0x7f57d40ad620 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53998/0 (socket says 172.21.15.98:53998) 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d40ad250 0x7f57d4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 -- 172.21.15.98:0/64269821 learned_addr learned my addr 172.21.15.98:0/64269821 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d37fe700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d40ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 -- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 msgr2=0x7f57d40ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d40ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 -- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d40ad250 msgr2=0x7f57d4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d40ad250 0x7f57d4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.901+0000 7f57d8c42700 1 -- 172.21.15.98:0/64269821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80094f0 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57d8c42700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c910 tx=0x7f57cc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc012630 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0127b0 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc012b70 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d40af5d0 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.902+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d40afb90 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.903+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d40b0070 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.903+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57cc01b400 con 0x7f57d415d980 2023-10-06T01:40:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.903+0000 7f57bbfff700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.904+0000 7f57d3fff700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.904+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f57cc084ce0 con 0x7f57d415d980 2023-10-06T01:40:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.904+0000 7f57d3fff700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 0x7f57c00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009d00 tx=0x7f57c4009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:24.908+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57cc051b00 con 0x7f57d415d980 2023-10-06T01:40:25.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.164+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) v1 -- 0x7f57d4064ad0 con 0x7f57d415d980 2023-10-06T01:40:25.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.182+0000 7f57bbfff700 1 -- 172.21.15.98:0/64269821 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v59) v1 ==== 150+0+0 (secure 0 0 0) 0x7f57cc055120 con 0x7f57d415d980 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 msgr2=0x7f57c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 0x7f57c00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009d00 tx=0x7f57c4009120 comp rx=0 tx=0).stop 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 msgr2=0x7f57d40aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c910 tx=0x7f57cc0027a0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 shutdown_connections 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d415d980 0x7f57d40aad20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d40ad250 0x7f57d4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 --2- 172.21.15.98:0/64269821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40ad620 0x7f57d40ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 >> 172.21.15.98:0/64269821 conn(0x7f57d4067dd0 msgr2=0x7f57d406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:25.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 shutdown_connections 2023-10-06T01:40:25.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.184+0000 7f57da6a5700 1 -- 172.21.15.98:0/64269821 wait complete. 2023-10-06T01:40:25.196 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T01:40:25.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.313+0000 7fb3faa0c700 1 Processor -- start 2023-10-06T01:40:25.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.313+0000 7fb3faa0c700 1 -- start start 2023-10-06T01:40:25.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3f40b2660 0x7fb3f40cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40b2a30 0x7fb3f40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3f40ceae0 0x7fb3f40cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40cc9b0 con 0x7fb3f40b2660 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40cf4c0 con 0x7fb3f40ceae0 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3faa0c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40cbda0 con 0x7fb3f40b2a30 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3f40b2660 0x7fb3f40cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3f40b2660 0x7fb3f40cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54012/0 (socket says 172.21.15.98:54012) 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 -- 172.21.15.98:0/879984629 learned_addr learned my addr 172.21.15.98:0/879984629 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 -- 172.21.15.98:0/879984629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40b2a30 msgr2=0x7fb3f40cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/879984629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40b2a30 0x7fb3f40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3fa20b700 1 --2- 172.21.15.98:0/879984629 >> v2:172.21.15.184:3300/0 conn(0x7fb3f40ceae0 0x7fb3f40cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.314+0000 7fb3f9a0a700 1 -- 172.21.15.98:0/879984629 >> v2:172.21.15.184:3300/0 conn(0x7fb3f40ceae0 msgr2=0x7fb3f40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/879984629 >> v2:172.21.15.184:3300/0 conn(0x7fb3f40ceae0 0x7fb3f40cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f9a0a700 1 -- 172.21.15.98:0/879984629 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40cc080 con 0x7fb3f40b2660 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/879984629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2660 0x7fb3f40cccf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4089690 tx=0x7fb3f0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f7f37d0746a573 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f8a08700 1 -- 172.21.15.98:0/879984629 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005c10 con 0x7fb3f40b2660 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f8a08700 1 -- 172.21.15.98:0/879984629 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0005d70 con 0x7fb3f40b2660 2023-10-06T01:40:25.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3f8a08700 1 -- 172.21.15.98:0/879984629 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000b880 con 0x7fb3f40b2660 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 -- 172.21.15.98:0/879984629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2660 msgr2=0x7fb3f40cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/879984629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2660 0x7fb3f40cccf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4089690 tx=0x7fb3f0009220 comp rx=0 tx=0).stop 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 -- 172.21.15.98:0/879984629 shutdown_connections 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/879984629 >> v2:172.21.15.184:3300/0 conn(0x7fb3f40ceae0 0x7fb3f40cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/879984629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2660 0x7fb3f40cccf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/879984629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40b2a30 0x7fb3f40cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.315+0000 7fb3faa0c700 1 -- 172.21.15.98:0/879984629 >> 172.21.15.98:0/879984629 conn(0x7fb3f4099560 msgr2=0x7fb3f406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.316+0000 7fb3faa0c700 1 -- 172.21.15.98:0/879984629 shutdown_connections 2023-10-06T01:40:25.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.316+0000 7fb3faa0c700 1 -- 172.21.15.98:0/879984629 wait complete. 2023-10-06T01:40:25.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 Processor -- start 2023-10-06T01:40:25.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 -- start start 2023-10-06T01:40:25.322 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f40b2660 0x7fb3f4157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40ceae0 0x7fb3f40b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f415a340 con 0x7fb3f40b2a30 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40c8560 con 0x7fb3f40b2660 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3faa0c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4158230 con 0x7fb3f40ceae0 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3f9209700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3f9209700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54018/0 (socket says 172.21.15.98:54018) 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.317+0000 7fb3f9209700 1 -- 172.21.15.98:0/3840005605 learned_addr learned my addr 172.21.15.98:0/3840005605 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:25.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3fa20b700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40ceae0 0x7fb3f40b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f40b2660 0x7fb3f4157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 -- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40ceae0 msgr2=0x7fb3f40b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40ceae0 0x7fb3f40b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 -- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f40b2660 msgr2=0x7fb3f4157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f40b2660 0x7fb3f4157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 -- 172.21.15.98:0/3840005605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0009480 con 0x7fb3f40b2a30 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3f9209700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b810 tx=0x7fb3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400f5f0 con 0x7fb3f40b2a30 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e400f770 con 0x7fb3f40b2a30 2023-10-06T01:40:25.324 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f4159d30 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400fb30 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.318+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f4159fd0 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.319+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f415a4b0 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.319+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3e4010480 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.320+0000 7fb3e9ffb700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 0x7fb3d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.320+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 0x7fb3d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.320+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb3e40848c0 con 0x7fb3f40b2a30 2023-10-06T01:40:25.325 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.320+0000 7fb3f9a0a700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 0x7fb3d80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009260 tx=0x7fb3f0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:25.326 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.324+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3e4051760 con 0x7fb3f40b2a30 2023-10-06T01:40:25.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.511+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3f405f270 con 0x7fb3f40b2a30 2023-10-06T01:40:25.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.515+0000 7fb3e9ffb700 1 -- 172.21.15.98:0/3840005605 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3e4054d80 con 0x7fb3f40b2a30 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 msgr2=0x7fb3d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 0x7fb3d80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009260 tx=0x7fb3f0016000 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 msgr2=0x7fb3f40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b810 tx=0x7fb3e4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 shutdown_connections 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3d8066f30 0x7fb3d80693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f40ceae0 0x7fb3f40b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f40b2660 0x7fb3f4157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 --2- 172.21.15.98:0/3840005605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3f40b2a30 0x7fb3f40b4330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:25.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 >> 172.21.15.98:0/3840005605 conn(0x7fb3f4099560 msgr2=0x7fb3f40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:25.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 shutdown_connections 2023-10-06T01:40:25.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:25.517+0000 7fb3faa0c700 1 -- 172.21.15.98:0/3840005605 wait complete. 2023-10-06T01:40:25.529 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T01:40:25.530 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T01:40:25.562 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T01:40:25.605 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T01:40:25.605 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T01:40:25.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:25.605 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T01:40:25.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:40:25.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:25.640 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T01:40:25.695 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T01:40:25.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:25.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T01:40:25.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:40:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:25.734 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T01:40:25.792 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T01:40:25.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:25.792 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T01:40:25.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:40:25.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:25.827 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T01:40:25.881 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T01:40:25.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:25.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T01:40:25.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:40:25.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:25.921 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T01:40:25.977 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T01:40:25.977 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:26.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.091+0000 7f6a01597700 1 Processor -- start 2023-10-06T01:40:26.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.091+0000 7f6a01597700 1 -- start start 2023-10-06T01:40:26.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f6a01597700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 0x7f69fc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f6a01597700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0cb7e0 con 0x7f69fc0cbc70 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f6a01597700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0b08a0 con 0x7f69fc159480 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f6a01597700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0cb190 con 0x7f69fc0b2850 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f69fa7fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f69fc0cbc70 0x7f69fc0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41780/0 (socket says 172.21.15.98:41780) 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f69fa7fc700 1 -- 172.21.15.98:0/4175224236 learned_addr learned my addr 172.21.15.98:0/4175224236 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f69fb7fe700 1 --2- 172.21.15.98:0/4175224236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 0x7f69fc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013853103 0 0) 0x7f69fc0cb7e0 con 0x7f69fc0cbc70 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.092+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e8003570 con 0x7f69fc0cbc70 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313630284 0 0) 0x7f69fc0b08a0 con 0x7f69fc159480 2023-10-06T01:40:26.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69fc0cb7e0 con 0x7f69fc159480 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4063030425 0 0) 0x7f69e8003570 con 0x7f69fc0cbc70 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69fb7fe700 1 -- 172.21.15.98:0/4175224236 >> v1:172.21.15.184:6789/0 conn(0x7f69fc159480 legacy=0x7f69fc0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69fb7fe700 1 -- 172.21.15.98:0/4175224236 >> v1:172.21.15.98:6789/0 conn(0x7f69fc0cbc70 legacy=0x7f69fc0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69fb7fe700 1 -- 172.21.15.98:0/4175224236 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0cb470 con 0x7f69fc0b2850 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69fb7fe700 1 --2- 172.21.15.98:0/4175224236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 0x7f69fc0cc0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009910 tx=0x7f69f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6706960c28b5ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4001470 con 0x7f69fc0b2850 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4011420 con 0x7f69fc0b2850 2023-10-06T01:40:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f69f9ffb700 1 -- 172.21.15.98:0/4175224236 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f40117c0 con 0x7f69fc0b2850 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f6a01597700 1 -- 172.21.15.98:0/4175224236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 msgr2=0x7f69fc0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.093+0000 7f6a01597700 1 --2- 172.21.15.98:0/4175224236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 0x7f69fc0cc0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009910 tx=0x7f69f40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.094+0000 7f6a01597700 1 -- 172.21.15.98:0/4175224236 shutdown_connections 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.094+0000 7f6a01597700 1 --2- 172.21.15.98:0/4175224236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0b2850 0x7f69fc0cc0c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.094+0000 7f6a01597700 1 -- 172.21.15.98:0/4175224236 >> 172.21.15.98:0/4175224236 conn(0x7f69fc06ed20 msgr2=0x7f69fc06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.094+0000 7f6a01597700 1 -- 172.21.15.98:0/4175224236 shutdown_connections 2023-10-06T01:40:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.094+0000 7f6a01597700 1 -- 172.21.15.98:0/4175224236 wait complete. 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.095+0000 7f6a01597700 1 Processor -- start 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.095+0000 7f6a01597700 1 -- start start 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69fc0b2850 0x7f69fc0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 0x7f69fc0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0c8e80 con 0x7f69fc159480 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc169150 con 0x7f69fc0b2850 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0c9a90 con 0x7f69fc0cbc70 2023-10-06T01:40:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54030/0 (socket says 172.21.15.98:54030) 2023-10-06T01:40:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 -- 172.21.15.98:0/2173312691 learned_addr learned my addr 172.21.15.98:0/2173312691 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fa7fc700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 0x7f69fc0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69faffd700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69fc0b2850 0x7f69fc0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 -- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 msgr2=0x7f69fc0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 0x7f69fc0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 -- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69fc0b2850 msgr2=0x7f69fc0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69fc0b2850 0x7f69fc0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 -- 172.21.15.98:0/2173312691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009420 con 0x7f69fc159480 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fa7fc700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 0x7f69fc0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69fb7fe700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009860 tx=0x7f69f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f401cdf0 con 0x7f69fc159480 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f401a450 con 0x7f69fc159480 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc0c8300 con 0x7f69fc159480 2023-10-06T01:40:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f401a810 con 0x7f69fc159480 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.096+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc0c88c0 con 0x7f69fc159480 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.097+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc1692c0 con 0x7f69fc159480 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.098+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69f401ac40 con 0x7f69fc159480 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.098+0000 7f69f2ffd700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.098+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f69f4086a70 con 0x7f69fc159480 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.098+0000 7f69faffd700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.098+0000 7f69faffd700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 0x7f69dc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69ec007fd0 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.102+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69f4053890 con 0x7f69fc159480 2023-10-06T01:40:26.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.330+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69fc05f270 con 0x7f69fc159480 2023-10-06T01:40:26.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.330+0000 7f69f2ffd700 1 -- 172.21.15.98:0/2173312691 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f69f4056eb0 con 0x7f69fc159480 2023-10-06T01:40:26.331 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:26.331 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:26.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 msgr2=0x7f69dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 0x7f69dc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69ec007fd0 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2023-10-06T01:40:26.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 msgr2=0x7f69fc1537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009860 tx=0x7f69f4016000 comp rx=0 tx=0).stop 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 shutdown_connections 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69fc159480 0x7f69fc1537d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69fc0b2850 0x7f69fc0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 --2- 172.21.15.98:0/2173312691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69fc0cbc70 0x7f69fc0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.332+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 >> 172.21.15.98:0/2173312691 conn(0x7f69fc06ed20 msgr2=0x7f69fc156cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.333+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 shutdown_connections 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.333+0000 7f6a01597700 1 -- 172.21.15.98:0/2173312691 wait complete. 2023-10-06T01:40:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:26.345 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:26.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.458+0000 7f43ca837700 1 Processor -- start 2023-10-06T01:40:26.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.458+0000 7f43ca837700 1 -- start start 2023-10-06T01:40:26.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43c4159480 0x7f43c40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43c40b3610 0x7f43c40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40cc1b0 0x7f43c40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c40cb7e0 con 0x7f43c40b3610 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c40b08a0 con 0x7f43c4159480 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca837700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40cb190 con 0x7f43c40cc1b0 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43c40b3610 0x7f43c40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43c40b3610 0x7f43c40cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54042/0 (socket says 172.21.15.98:54042) 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 -- 172.21.15.98:0/3290698722 learned_addr learned my addr 172.21.15.98:0/3290698722 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9835700 1 --2- 172.21.15.98:0/3290698722 >> v2:172.21.15.184:3300/0 conn(0x7f43c4159480 0x7f43c40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43ca036700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40cc1b0 0x7f43c40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 -- 172.21.15.98:0/3290698722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40cc1b0 msgr2=0x7f43c40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40cc1b0 0x7f43c40b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 -- 172.21.15.98:0/3290698722 >> v2:172.21.15.184:3300/0 conn(0x7f43c4159480 msgr2=0x7f43c40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.459+0000 7f43c9034700 1 --2- 172.21.15.98:0/3290698722 >> v2:172.21.15.184:3300/0 conn(0x7f43c4159480 0x7f43c40b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43c9034700 1 -- 172.21.15.98:0/3290698722 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40cb470 con 0x7f43c40b3610 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43c9034700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40cbc70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43c406fa30 tx=0x7f43b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3218c645a89f9746 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43c8833700 1 -- 172.21.15.98:0/3290698722 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400b860 con 0x7f43c40b3610 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43c8833700 1 -- 172.21.15.98:0/3290698722 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b400b9c0 con 0x7f43c40b3610 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43c8833700 1 -- 172.21.15.98:0/3290698722 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400bd60 con 0x7f43c40b3610 2023-10-06T01:40:26.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 -- 172.21.15.98:0/3290698722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 msgr2=0x7f43c40cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40cbc70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43c406fa30 tx=0x7f43b4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 -- 172.21.15.98:0/3290698722 shutdown_connections 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 --2- 172.21.15.98:0/3290698722 >> v2:172.21.15.184:3300/0 conn(0x7f43c4159480 0x7f43c40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40cc1b0 0x7f43c40b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 --2- 172.21.15.98:0/3290698722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40cbc70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 -- 172.21.15.98:0/3290698722 >> 172.21.15.98:0/3290698722 conn(0x7f43c406ed20 msgr2=0x7f43c406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 -- 172.21.15.98:0/3290698722 shutdown_connections 2023-10-06T01:40:26.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.460+0000 7f43ca837700 1 -- 172.21.15.98:0/3290698722 wait complete. 2023-10-06T01:40:26.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 Processor -- start 2023-10-06T01:40:26.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 -- start start 2023-10-06T01:40:26.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40cc1b0 0x7f43c40c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 0x7f43c40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c6380 con 0x7f43c40b3610 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40cec10 con 0x7f43c40cc1b0 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca837700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4159cd0 con 0x7f43c4159480 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43c9835700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43c9835700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54054/0 (socket says 172.21.15.98:54054) 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43c9835700 1 -- 172.21.15.98:0/1084736018 learned_addr learned my addr 172.21.15.98:0/1084736018 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43c9034700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40cc1b0 0x7f43c40c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.462+0000 7f43ca036700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 0x7f43c40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 -- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 msgr2=0x7f43c40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 0x7f43c40cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 -- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40cc1b0 msgr2=0x7f43c40c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40cc1b0 0x7f43c40c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 -- 172.21.15.98:0/1084736018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40094f0 con 0x7f43c40b3610 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43ca036700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 0x7f43c40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43c9835700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c8b0 tx=0x7f43c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0011600 con 0x7f43c40b3610 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0011760 con 0x7f43c40b3610 2023-10-06T01:40:26.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c40ce8e0 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0011b00 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.463+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c40c68c0 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.464+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c40ceef0 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.464+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43c0019400 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.465+0000 7f43b9ffb700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.465+0000 7f43c9034700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.465+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f43c0084980 con 0x7f43c40b3610 2023-10-06T01:40:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.465+0000 7f43c9034700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 0x7f43a80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43c406fa30 tx=0x7f43b4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.469+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43c00517a0 con 0x7f43c40b3610 2023-10-06T01:40:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.653+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c40c6be0 con 0x7f43c40b3610 2023-10-06T01:40:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.653+0000 7f43b9ffb700 1 -- 172.21.15.98:0/1084736018 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f43c0054dc0 con 0x7f43c40b3610 2023-10-06T01:40:26.655 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:26.655 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 msgr2=0x7f43a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 0x7f43a80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43c406fa30 tx=0x7f43b4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 msgr2=0x7f43c40c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c8b0 tx=0x7f43c0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 shutdown_connections 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4159480 0x7f43c40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43c40b3610 0x7f43c40c3d60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 --2- 172.21.15.98:0/1084736018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40cc1b0 0x7f43c40c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 >> 172.21.15.98:0/1084736018 conn(0x7f43c406ed20 msgr2=0x7f43c4152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 shutdown_connections 2023-10-06T01:40:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.656+0000 7f43ca837700 1 -- 172.21.15.98:0/1084736018 wait complete. 2023-10-06T01:40:26.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:39:42.361455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:39:43.656625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T01:39:47.351521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:40:26.670 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:26.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.788+0000 7f913196f700 1 Processor -- start 2023-10-06T01:40:26.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.788+0000 7f913196f700 1 -- start start 2023-10-06T01:40:26.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f913196f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 0x7f912c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f913196f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0b1970 con 0x7f912c1568f0 2023-10-06T01:40:26.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f913196f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0b0d60 con 0x7f912c156520 2023-10-06T01:40:26.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f913196f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0b1040 con 0x7f912c0b08d0 2023-10-06T01:40:26.791 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f912b7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f912c1568f0 0x7f912c0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41782/0 (socket says 172.21.15.98:41782) 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f912b7fe700 1 -- 172.21.15.98:0/2973602647 learned_addr learned my addr 172.21.15.98:0/2973602647 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f91227fc700 1 --2- 172.21.15.98:0/2973602647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 0x7f912c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116438159 0 0) 0x7f912c0b1970 con 0x7f912c1568f0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.789+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9118003570 con 0x7f912c1568f0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368961990 0 0) 0x7f912c0b0d60 con 0x7f912c156520 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912c0b1970 con 0x7f912c156520 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f91227fc700 1 -- 172.21.15.98:0/2973602647 >> v1:172.21.15.184:6789/0 conn(0x7f912c156520 legacy=0x7f912c06ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556563393 0 0) 0x7f9118003570 con 0x7f912c1568f0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f91227fc700 1 -- 172.21.15.98:0/2973602647 >> v1:172.21.15.98:6789/0 conn(0x7f912c1568f0 legacy=0x7f912c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f91227fc700 1 -- 172.21.15.98:0/2973602647 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0b1320 con 0x7f912c0b08d0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f91227fc700 1 --2- 172.21.15.98:0/2973602647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 0x7f912c06e310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9114009910 tx=0x7f91140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a053499f76c2569a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.792 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240017f0 con 0x7f912c0b08d0 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114011420 con 0x7f912c0b08d0 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f912a7fc700 1 -- 172.21.15.98:0/2973602647 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140117c0 con 0x7f912c0b08d0 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f913196f700 1 -- 172.21.15.98:0/2973602647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 msgr2=0x7f912c06e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.790+0000 7f913196f700 1 --2- 172.21.15.98:0/2973602647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 0x7f912c06e310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9114009910 tx=0x7f91140091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.791+0000 7f913196f700 1 -- 172.21.15.98:0/2973602647 shutdown_connections 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.791+0000 7f913196f700 1 --2- 172.21.15.98:0/2973602647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c0b08d0 0x7f912c06e310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.791+0000 7f913196f700 1 -- 172.21.15.98:0/2973602647 >> 172.21.15.98:0/2973602647 conn(0x7f912c069190 msgr2=0x7f912c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.791+0000 7f913196f700 1 -- 172.21.15.98:0/2973602647 shutdown_connections 2023-10-06T01:40:26.793 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.791+0000 7f913196f700 1 -- 172.21.15.98:0/2973602647 wait complete. 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 Processor -- start 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 -- start start 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c0b08d0 0x7f912c13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c156520 0x7f912c152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c148300 con 0x7f912c1568f0 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c14f400 con 0x7f912c0b08d0 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.792+0000 7f913196f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c14e7f0 con 0x7f912c156520 2023-10-06T01:40:26.796 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54060/0 (socket says 172.21.15.98:54060) 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 -- 172.21.15.98:0/4031259318 learned_addr learned my addr 172.21.15.98:0/4031259318 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 -- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c156520 msgr2=0x7f912c152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f91227fc700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c156520 0x7f912c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912affd700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c0b08d0 0x7f912c13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c156520 0x7f912c152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 -- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c0b08d0 msgr2=0x7f912c13dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c0b08d0 0x7f912c13dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 -- 172.21.15.98:0/4031259318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114009420 con 0x7f912c1568f0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f912b7fe700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f912400eed0 tx=0x7f9124007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124004bb0 con 0x7f912c1568f0 2023-10-06T01:40:26.797 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9124004d50 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c14ead0 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240073a0 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.793+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c14f090 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.794+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c05f270 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.794+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9124007790 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.795+0000 7f91237fe700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.795+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f91240874f0 con 0x7f912c1568f0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.795+0000 7f912affd700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:26.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.795+0000 7f912affd700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 0x7f910c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f911c0027d0 tx=0x7f911c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:26.799 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.799+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9124054310 con 0x7f912c1568f0 2023-10-06T01:40:26.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.980+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f912c0cbe20 con 0x7f912c1568f0 2023-10-06T01:40:26.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.981+0000 7f91237fe700 1 -- 172.21.15.98:0/4031259318 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9124007a40 con 0x7f912c1568f0 2023-10-06T01:40:26.982 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:26.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 msgr2=0x7f910c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 0x7f910c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f911c0027d0 tx=0x7f911c009040 comp rx=0 tx=0).stop 2023-10-06T01:40:26.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 msgr2=0x7f912c152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:26.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f912400eed0 tx=0x7f9124007da0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 shutdown_connections 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f912c1568f0 0x7f912c152950 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c0b08d0 0x7f912c13dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 --2- 172.21.15.98:0/4031259318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f912c156520 0x7f912c152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 >> 172.21.15.98:0/4031259318 conn(0x7f912c069190 msgr2=0x7f912c154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 shutdown_connections 2023-10-06T01:40:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:26.983+0000 7f913196f700 1 -- 172.21.15.98:0/4031259318 wait complete. 2023-10-06T01:40:26.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.112+0000 7fe5db16f700 1 Processor -- start 2023-10-06T01:40:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.112+0000 7fe5db16f700 1 -- start start 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5d4159340 0x7fe5d40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5d4159710 0x7fe5d40cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 0x7fe5d40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40cbed0 con 0x7fe5d4159710 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40cf820 con 0x7fe5d4159340 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5db16f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40cfb00 con 0x7fe5d40cba70 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5cbfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5d4159710 0x7fe5d40cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.116 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5cbfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5d4159710 0x7fe5d40cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54066/0 (socket says 172.21.15.98:54066) 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.113+0000 7fe5cbfff700 1 -- 172.21.15.98:0/3682177099 learned_addr learned my addr 172.21.15.98:0/3682177099 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5d8f0b700 1 --2- 172.21.15.98:0/3682177099 >> v2:172.21.15.184:3300/0 conn(0x7fe5d4159340 0x7fe5d40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5d970c700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 0x7fe5d40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 -- 172.21.15.98:0/3682177099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 msgr2=0x7fe5d40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 0x7fe5d40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 -- 172.21.15.98:0/3682177099 >> v2:172.21.15.184:3300/0 conn(0x7fe5d4159340 msgr2=0x7fe5d40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 --2- 172.21.15.98:0/3682177099 >> v2:172.21.15.184:3300/0 conn(0x7fe5d4159340 0x7fe5d40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 -- 172.21.15.98:0/3682177099 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40cfde0 con 0x7fe5d4159710 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5d970c700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 0x7fe5d40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cbfff700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d40cb530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4071290 tx=0x7fe5c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1240085a0af39fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.117 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cb7fe700 1 -- 172.21.15.98:0/3682177099 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400b860 con 0x7fe5d4159710 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cb7fe700 1 -- 172.21.15.98:0/3682177099 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c400b9c0 con 0x7fe5d4159710 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5cb7fe700 1 -- 172.21.15.98:0/3682177099 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400bd60 con 0x7fe5d4159710 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 -- 172.21.15.98:0/3682177099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 msgr2=0x7fe5d40cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d40cb530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4071290 tx=0x7fe5c4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 -- 172.21.15.98:0/3682177099 shutdown_connections 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3682177099 >> v2:172.21.15.184:3300/0 conn(0x7fe5d4159340 0x7fe5d40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d40cba70 0x7fe5d40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3682177099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d40cb530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.114+0000 7fe5db16f700 1 -- 172.21.15.98:0/3682177099 >> 172.21.15.98:0/3682177099 conn(0x7fe5d4099720 msgr2=0x7fe5d406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.115+0000 7fe5db16f700 1 -- 172.21.15.98:0/3682177099 shutdown_connections 2023-10-06T01:40:27.118 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.115+0000 7fe5db16f700 1 -- 172.21.15.98:0/3682177099 wait complete. 2023-10-06T01:40:27.119 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.116+0000 7fe5db16f700 1 Processor -- start 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.116+0000 7fe5db16f700 1 -- start start 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5d40cba70 0x7fe5d4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d4159340 0x7fe5d414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d415a4c0 con 0x7fe5d4159710 2023-10-06T01:40:27.121 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d414e2e0 con 0x7fe5d40cba70 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4156ef0 con 0x7fe5d4159340 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54078/0 (socket says 172.21.15.98:54078) 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 -- 172.21.15.98:0/3943093332 learned_addr learned my addr 172.21.15.98:0/3943093332 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d8f0b700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5d40cba70 0x7fe5d4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5cbfff700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d4159340 0x7fe5d414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 -- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d4159340 msgr2=0x7fe5d414d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d4159340 0x7fe5d414d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 -- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5d40cba70 msgr2=0x7fe5d4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5d40cba70 0x7fe5d4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 -- 172.21.15.98:0/3943093332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40094f0 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5d970c700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00a380 tx=0x7fe5cc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc00ae60 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc015440 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d4158470 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc015800 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.117+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4157240 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.118+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d4157da0 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.119+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5cc00d440 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.119+0000 7fe5c8ff9700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 0x7fe5c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.119+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe5cc085580 con 0x7fe5d4159710 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.119+0000 7fe5d8f0b700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 0x7fe5c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.123 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.119+0000 7fe5d8f0b700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 0x7fe5c00693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006aa0 tx=0x7fe5d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.123+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5cc0523a0 con 0x7fe5d4159710 2023-10-06T01:40:27.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.307+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5d4064b30 con 0x7fe5d4159710 2023-10-06T01:40:27.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.307+0000 7fe5c8ff9700 1 -- 172.21.15.98:0/3943093332 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5cc0559c0 con 0x7fe5d4159710 2023-10-06T01:40:27.308 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:27.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 msgr2=0x7fe5c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 0x7fe5c00693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006aa0 tx=0x7fe5d0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 msgr2=0x7fe5d414dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00a380 tx=0x7fe5cc007900 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 shutdown_connections 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe5c0066f30 0x7fe5c00693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5d4159710 0x7fe5d414dda0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5d40cba70 0x7fe5d4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 --2- 172.21.15.98:0/3943093332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5d4159340 0x7fe5d414d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 >> 172.21.15.98:0/3943093332 conn(0x7fe5d4099720 msgr2=0x7fe5d414f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 shutdown_connections 2023-10-06T01:40:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.310+0000 7fe5db16f700 1 -- 172.21.15.98:0/3943093332 wait complete. 2023-10-06T01:40:27.322 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:40:27.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.439+0000 7fcb11b12700 1 Processor -- start 2023-10-06T01:40:27.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.439+0000 7fcb11b12700 1 -- start start 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb11b12700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 0x7fcb0c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb11b12700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb0c0cba70 0x7fcb0c0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb11b12700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c0afbe0 con 0x7fcb0c159340 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb11b12700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cbf10 con 0x7fcb0c0cba70 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb11b12700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cf820 con 0x7fcb0c159710 2023-10-06T01:40:27.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb0c0cba70 0x7fcb0c0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 0x7fcb0c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0b7fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fcb0c159340 0x7fcb0c0af4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:48436/0 (socket says 172.21.15.98:48436) 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb0c0cba70 0x7fcb0c0c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:35598/0 (socket says 172.21.15.98:35598) 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 0x7fcb0c0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:33024/0 (socket says 172.21.15.98:33024) 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.440+0000 7fcb0b7fe700 1 -- 172.21.15.98:0/3445250175 learned_addr learned my addr 172.21.15.98:0/3445250175 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0bfff700 1 -- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 msgr2=0x7fcb0c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0bfff700 1 --2- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 0x7fcb0c0cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0bfff700 1 -- 172.21.15.98:0/3445250175 >> v1:172.21.15.184:6789/0 conn(0x7fcb0c159340 legacy=0x7fcb0c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0bfff700 1 -- 172.21.15.98:0/3445250175 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0cfb00 con 0x7fcb0c0cba70 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0a7fc700 1 -- 172.21.15.98:0/3445250175 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814830188 0 0) 0x7fcafc0014e0 con 0x7fcb0c159340 2023-10-06T01:40:27.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0bfff700 1 --2- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c0cba70 0x7fcb0c0c5660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c62fb043b273f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0a7fc700 1 -- 172.21.15.98:0/3445250175 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400b640 con 0x7fcb0c0cba70 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0a7fc700 1 -- 172.21.15.98:0/3445250175 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400b7a0 con 0x7fcb0c0cba70 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb0a7fc700 1 -- 172.21.15.98:0/3445250175 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400bb40 con 0x7fcb0c0cba70 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 -- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c0cba70 msgr2=0x7fcb0c0c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 --2- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c0cba70 0x7fcb0c0c5660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 -- 172.21.15.98:0/3445250175 shutdown_connections 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 --2- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c0cba70 0x7fcb0c0c5660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 --2- 172.21.15.98:0/3445250175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c159710 0x7fcb0c0cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.441+0000 7fcb11b12700 1 -- 172.21.15.98:0/3445250175 >> 172.21.15.98:0/3445250175 conn(0x7fcb0c099720 msgr2=0x7fcb0c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.442+0000 7fcb11b12700 1 -- 172.21.15.98:0/3445250175 shutdown_connections 2023-10-06T01:40:27.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.442+0000 7fcb11b12700 1 -- 172.21.15.98:0/3445250175 wait complete. 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 Processor -- start 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 -- start start 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c0cba70 0x7fcb0c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c159340 0x7fcb0c14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c14aab0 con 0x7fcb0c159710 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15dd90 con 0x7fcb0c159340 2023-10-06T01:40:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.443+0000 7fcb11b12700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15cea0 con 0x7fcb0c0cba70 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54092/0 (socket says 172.21.15.98:54092) 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 -- 172.21.15.98:0/2698428411 learned_addr learned my addr 172.21.15.98:0/2698428411 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0affd700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c159340 0x7fcb0c14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0b7fe700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c0cba70 0x7fcb0c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 -- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c0cba70 msgr2=0x7fcb0c0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c0cba70 0x7fcb0c0ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 -- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c159340 msgr2=0x7fcb0c14ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c159340 0x7fcb0c14ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 -- 172.21.15.98:0/2698428411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04009420 con 0x7fcb0c159710 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb0bfff700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0712c0 tx=0x7fcb04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400bb20 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04011420 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c15d180 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040117a0 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.444+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c15d740 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.445+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c15df00 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.446+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb04011b10 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.446+0000 7fcaf37fe700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.446+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcb04085e40 con 0x7fcb0c159710 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.446+0000 7fcb0b7fe700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.446+0000 7fcb0b7fe700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007300 tx=0x7fcafc001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.450+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb04052c60 con 0x7fcb0c159710 2023-10-06T01:40:27.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.631+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb0c064b30 con 0x7fcb0c159710 2023-10-06T01:40:27.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.632+0000 7fcaf37fe700 1 -- 172.21.15.98:0/2698428411 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb04056280 con 0x7fcb0c159710 2023-10-06T01:40:27.633 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 msgr2=0x7fcaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007300 tx=0x7fcafc001a00 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 msgr2=0x7fcb0c149b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0712c0 tx=0x7fcb04017000 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 shutdown_connections 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb0c159710 0x7fcb0c149b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb0c0cba70 0x7fcb0c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 --2- 172.21.15.98:0/2698428411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0c159340 0x7fcb0c14ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 >> 172.21.15.98:0/2698428411 conn(0x7fcb0c099720 msgr2=0x7fcb0c14bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 shutdown_connections 2023-10-06T01:40:27.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.634+0000 7fcb11b12700 1 -- 172.21.15.98:0/2698428411 wait complete. 2023-10-06T01:40:27.646 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:40:27.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.762+0000 7fa0da2f6700 1 Processor -- start 2023-10-06T01:40:27.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.762+0000 7fa0da2f6700 1 -- start start 2023-10-06T01:40:27.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0da2f6700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0d40b2660 0x7fa0d40cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.765 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0da2f6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 0x7fa0d40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0da2f6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d40cc9b0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0da2f6700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d40cbda0 con 0x7fa0d40b2660 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0da2f6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40cc080 con 0x7fa0d40b2a30 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d92f4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0d40b2660 0x7fa0d40cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d8af3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 0x7fa0d40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d92f4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0d40b2660 0x7fa0d40cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:35608/0 (socket says 172.21.15.98:35608) 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d8af3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 0x7fa0d40cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:33042/0 (socket says 172.21.15.98:33042) 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d92f4700 1 -- 172.21.15.98:0/1012301622 learned_addr learned my addr 172.21.15.98:0/1012301622 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.763+0000 7fa0d9af5700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa0d40ceae0 0x7fa0d40cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:48448/0 (socket says 172.21.15.98:48448) 2023-10-06T01:40:27.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0d92f4700 1 -- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 msgr2=0x7fa0d40cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0d92f4700 1 --2- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 0x7fa0d40cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0d92f4700 1 -- 172.21.15.98:0/1012301622 >> v1:172.21.15.184:6789/0 conn(0x7fa0d40ceae0 legacy=0x7fa0d40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0cbfff700 1 -- 172.21.15.98:0/1012301622 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764598593 0 0) 0x7fa0d40cc9b0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0d92f4700 1 -- 172.21.15.98:0/1012301622 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40cc360 con 0x7fa0d40b2660 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0d92f4700 1 --2- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2660 0x7fa0d40cccf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009990 tx=0x7fa0d0009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5607fe5b7579b7cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0cbfff700 1 -- 172.21.15.98:0/1012301622 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0009e80 con 0x7fa0d40b2660 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0cbfff700 1 -- 172.21.15.98:0/1012301622 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0005c10 con 0x7fa0d40b2660 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.764+0000 7fa0cbfff700 1 -- 172.21.15.98:0/1012301622 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000b650 con 0x7fa0d40b2660 2023-10-06T01:40:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 -- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2660 msgr2=0x7fa0d40cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2660 0x7fa0d40cccf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009990 tx=0x7fa0d0009220 comp rx=0 tx=0).stop 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 -- 172.21.15.98:0/1012301622 shutdown_connections 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2660 0x7fa0d40cccf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/1012301622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2a30 0x7fa0d40cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 -- 172.21.15.98:0/1012301622 >> 172.21.15.98:0/1012301622 conn(0x7fa0d4099560 msgr2=0x7fa0d406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 -- 172.21.15.98:0/1012301622 shutdown_connections 2023-10-06T01:40:27.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.765+0000 7fa0da2f6700 1 -- 172.21.15.98:0/1012301622 wait complete. 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.766+0000 7fa0da2f6700 1 Processor -- start 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.766+0000 7fa0da2f6700 1 -- start start 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2660 0x7fa0d40b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 0x7fa0d40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4153df0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40b3da0 con 0x7fa0d40b2a30 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0da2f6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d41537e0 con 0x7fa0d40b2660 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54094/0 (socket says 172.21.15.98:54094) 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 -- 172.21.15.98:0/3685581895 learned_addr learned my addr 172.21.15.98:0/3685581895 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d8af3700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 0x7fa0d40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d92f4700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2660 0x7fa0d40b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 -- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2660 msgr2=0x7fa0d40b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2660 0x7fa0d40b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 -- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 msgr2=0x7fa0d40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 0x7fa0d40b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 -- 172.21.15.98:0/3685581895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d0009480 con 0x7fa0d40ceae0 2023-10-06T01:40:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d8af3700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 0x7fa0d40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0d9af5700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00cd50 tx=0x7fa0cc001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc005df0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.767+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00a960 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.768+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40b37e0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.768+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00ad20 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.768+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d4153a80 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.768+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d405f270 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.769+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0cc007300 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.769+0000 7fa0c97fa700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 0x7fa0b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.769+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa0cc084bb0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.769+0000 7fa0d92f4700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 0x7fa0b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:27.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.770+0000 7fa0d92f4700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 0x7fa0b80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009260 tx=0x7fa0d0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:27.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.773+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0cc0519d0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.956 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.956+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0d40cc640 con 0x7fa0d40ceae0 2023-10-06T01:40:27.957 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.956+0000 7fa0c97fa700 1 -- 172.21.15.98:0/3685581895 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0cc054ff0 con 0x7fa0d40ceae0 2023-10-06T01:40:27.957 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:40:27.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 msgr2=0x7fa0b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 0x7fa0b80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009260 tx=0x7fa0d0016000 comp rx=0 tx=0).stop 2023-10-06T01:40:27.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 msgr2=0x7fa0d4154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:27.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00cd50 tx=0x7fa0cc001b10 comp rx=0 tx=0).stop 2023-10-06T01:40:27.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 shutdown_connections 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa0b8066f30 0x7fa0b80693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0d40ceae0 0x7fa0d4154280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d40b2660 0x7fa0d40b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 --2- 172.21.15.98:0/3685581895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d40b2a30 0x7fa0d40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 >> 172.21.15.98:0/3685581895 conn(0x7fa0d4099560 msgr2=0x7fa0d40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 shutdown_connections 2023-10-06T01:40:27.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:27.958+0000 7fa0da2f6700 1 -- 172.21.15.98:0/3685581895 wait complete. 2023-10-06T01:40:27.971 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T01:40:28.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.087+0000 7f5f64c56700 1 Processor -- start 2023-10-06T01:40:28.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.087+0000 7f5f64c56700 1 -- start start 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f64c56700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f600cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f64c56700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600cbf00 con 0x7f5f600cbae0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f64c56700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600cf610 con 0x7f5f601598e0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f64c56700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600cf8f0 con 0x7f5f60159510 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f5ed9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5f600cbae0 0x7f5f600c56d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41786/0 (socket says 172.21.15.98:41786) 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.088+0000 7f5f5ed9d700 1 -- 172.21.15.98:0/2748800413 learned_addr learned my addr 172.21.15.98:0/2748800413 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5e59c700 1 --2- 172.21.15.98:0/2748800413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f600cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307657120 0 0) 0x7f5f600cbf00 con 0x7f5f600cbae0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f4c003570 con 0x7f5f600cbae0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129833160 0 0) 0x7f5f600cf610 con 0x7f5f601598e0 2023-10-06T01:40:28.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f600cbf00 con 0x7f5f601598e0 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5e59c700 1 -- 172.21.15.98:0/2748800413 >> v1:172.21.15.184:6789/0 conn(0x7f5f601598e0 legacy=0x7f5f600ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5e59c700 1 -- 172.21.15.98:0/2748800413 >> v1:172.21.15.98:6789/0 conn(0x7f5f600cbae0 legacy=0x7f5f600c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5e59c700 1 -- 172.21.15.98:0/2748800413 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600cfbd0 con 0x7f5f60159510 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5e59c700 1 --2- 172.21.15.98:0/2748800413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f600cb5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009910 tx=0x7f5f500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6003a46b9e885f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50011630 con 0x7f5f60159510 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50011790 con 0x7f5f60159510 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.089+0000 7f5f5dd9b700 1 -- 172.21.15.98:0/2748800413 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50011b30 con 0x7f5f60159510 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 -- 172.21.15.98:0/2748800413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 msgr2=0x7f5f600cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2748800413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f600cb5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009910 tx=0x7f5f500091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 -- 172.21.15.98:0/2748800413 shutdown_connections 2023-10-06T01:40:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2748800413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f600cb5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 -- 172.21.15.98:0/2748800413 >> 172.21.15.98:0/2748800413 conn(0x7f5f60089e90 msgr2=0x7f5f600997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 -- 172.21.15.98:0/2748800413 shutdown_connections 2023-10-06T01:40:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.090+0000 7f5f64c56700 1 -- 172.21.15.98:0/2748800413 wait complete. 2023-10-06T01:40:28.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.091+0000 7f5f64c56700 1 Processor -- start 2023-10-06T01:40:28.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.091+0000 7f5f64c56700 1 -- start start 2023-10-06T01:40:28.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f600cbae0 0x7f5f600c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f6009d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600c9a70 con 0x7f5f601598e0 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6014ecb0 con 0x7f5f600cbae0 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f64c56700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600c8e60 con 0x7f5f60159510 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54102/0 (socket says 172.21.15.98:54102) 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 -- 172.21.15.98:0/2536335974 learned_addr learned my addr 172.21.15.98:0/2536335974 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f55d9b700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f6009d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5e59c700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f600cbae0 0x7f5f600c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 -- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 msgr2=0x7f5f6009d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f6009d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 -- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f600cbae0 msgr2=0x7f5f600c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f600cbae0 0x7f5f600c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 -- 172.21.15.98:0/2536335974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50009420 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f55d9b700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f6009d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.092+0000 7f5f5ed9d700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800cc60 tx=0x7f5f58001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.093+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800ce70 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.093+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800ab70 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.093+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f600c9140 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.093+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f580070f0 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.093+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f600c9700 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.094+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f60064ad0 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.094+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f580074a0 con 0x7f5f601598e0 2023-10-06T01:40:28.097 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.094+0000 7f5f56ffd700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 0x7f5f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.094+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5f58084e80 con 0x7f5f601598e0 2023-10-06T01:40:28.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.094+0000 7f5f5e59c700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 0x7f5f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.095+0000 7f5f5e59c700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 0x7f5f400693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009860 tx=0x7f5f50017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.098+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f580557b0 con 0x7f5f601598e0 2023-10-06T01:40:28.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.281+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f6014edf0 con 0x7f5f601598e0 2023-10-06T01:40:28.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.281+0000 7f5f56ffd700 1 -- 172.21.15.98:0/2536335974 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f6014edf0 con 0x7f5f601598e0 2023-10-06T01:40:28.282 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:40:28.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 msgr2=0x7f5f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 0x7f5f400693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009860 tx=0x7f5f50017000 comp rx=0 tx=0).stop 2023-10-06T01:40:28.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 msgr2=0x7f5f6014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800cc60 tx=0x7f5f58001ae0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 shutdown_connections 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5f40066f30 0x7f5f400693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f601598e0 0x7f5f6014e770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f600cbae0 0x7f5f600c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 --2- 172.21.15.98:0/2536335974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f60159510 0x7f5f6009d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 >> 172.21.15.98:0/2536335974 conn(0x7f5f60089e90 msgr2=0x7f5f60068690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 shutdown_connections 2023-10-06T01:40:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.283+0000 7f5f64c56700 1 -- 172.21.15.98:0/2536335974 wait complete. 2023-10-06T01:40:28.296 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:28.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.409+0000 7f140fa72700 1 Processor -- start 2023-10-06T01:40:28.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 -- start start 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ca3b0 0x7f14081582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 0x7f14080ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14080ce710 0x7f140815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14080cbbd0 con 0x7f14080ce710 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1408157e20 con 0x7f14080adba0 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140fa72700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080ceb30 con 0x7f14080ca3b0 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.410+0000 7f140e00f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14080ce710 0x7f140815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14080ce710 0x7f140815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54108/0 (socket says 172.21.15.98:54108) 2023-10-06T01:40:28.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 -- 172.21.15.98:0/2238024095 learned_addr learned my addr 172.21.15.98:0/2238024095 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140d80e700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ca3b0 0x7f14081582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140d00d700 1 --2- 172.21.15.98:0/2238024095 >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 0x7f14080ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 -- 172.21.15.98:0/2238024095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ca3b0 msgr2=0x7f14081582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ca3b0 0x7f14081582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 -- 172.21.15.98:0/2238024095 >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 msgr2=0x7f14080ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 --2- 172.21.15.98:0/2238024095 >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 0x7f14080ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 -- 172.21.15.98:0/2238024095 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080cb610 con 0x7f14080ce710 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140d00d700 1 --2- 172.21.15.98:0/2238024095 >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 0x7f14080ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140e00f700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080ce710 0x7f140815af10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f140806f260 tx=0x7f1400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36f81246d54cde4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140c80c700 1 -- 172.21.15.98:0/2238024095 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140000b860 con 0x7f14080ce710 2023-10-06T01:40:28.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140c80c700 1 -- 172.21.15.98:0/2238024095 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140000b9c0 con 0x7f14080ce710 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.411+0000 7f140c80c700 1 -- 172.21.15.98:0/2238024095 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140000bd60 con 0x7f14080ce710 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 -- 172.21.15.98:0/2238024095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080ce710 msgr2=0x7f140815af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080ce710 0x7f140815af10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f140806f260 tx=0x7f1400009250 comp rx=0 tx=0).stop 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 -- 172.21.15.98:0/2238024095 shutdown_connections 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080ce710 0x7f140815af10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 --2- 172.21.15.98:0/2238024095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ca3b0 0x7f14081582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 --2- 172.21.15.98:0/2238024095 >> v2:172.21.15.184:3300/0 conn(0x7f14080adba0 0x7f14080ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 -- 172.21.15.98:0/2238024095 >> 172.21.15.98:0/2238024095 conn(0x7f1408070810 msgr2=0x7f1408070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 -- 172.21.15.98:0/2238024095 shutdown_connections 2023-10-06T01:40:28.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.412+0000 7f140fa72700 1 -- 172.21.15.98:0/2238024095 wait complete. 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.413+0000 7f140fa72700 1 Processor -- start 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.413+0000 7f140fa72700 1 -- start start 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.413+0000 7f140fa72700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 0x7f1408148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ce710 0x7f1408159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080cb170 con 0x7f14080adba0 2023-10-06T01:40:28.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c1d50 con 0x7f14080ca3b0 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14081599e0 con 0x7f14080ce710 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54114/0 (socket says 172.21.15.98:54114) 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 -- 172.21.15.98:0/2300092659 learned_addr learned my addr 172.21.15.98:0/2300092659 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140e00f700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ce710 0x7f1408159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d00d700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 0x7f1408148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 -- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ce710 msgr2=0x7f1408159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ce710 0x7f1408159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 -- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 msgr2=0x7f1408148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 0x7f1408148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 -- 172.21.15.98:0/2300092659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000094f0 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d00d700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 0x7f1408148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140d80e700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f140400e1c0 tx=0x7f1404009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400edb0 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404014420 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1408069160 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14040147e0 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.414+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1408068830 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.415+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1408069440 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.415+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1404014b90 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.416+0000 7f13fdffb700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.416+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1404085120 con 0x7f14080adba0 2023-10-06T01:40:28.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.416+0000 7f140d00d700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.416+0000 7f140d00d700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 0x7f13ec0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14080cadf0 tx=0x7f13f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.420+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1404051f40 con 0x7f14080adba0 2023-10-06T01:40:28.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.644+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1408064ad0 con 0x7f14080adba0 2023-10-06T01:40:28.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.645+0000 7f13fdffb700 1 -- 172.21.15.98:0/2300092659 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f1404009a60 con 0x7f14080adba0 2023-10-06T01:40:28.646 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:28.646 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 msgr2=0x7f13ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 0x7f13ec0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14080cadf0 tx=0x7f13f8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 msgr2=0x7f14080b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f140400e1c0 tx=0x7f1404009920 comp rx=0 tx=0).stop 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 shutdown_connections 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f13ec066f30 0x7f13ec0693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14080ce710 0x7f1408159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080adba0 0x7f14080b3a10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 --2- 172.21.15.98:0/2300092659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14080ca3b0 0x7f1408148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 >> 172.21.15.98:0/2300092659 conn(0x7f1408070810 msgr2=0x7f1408157170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 shutdown_connections 2023-10-06T01:40:28.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.647+0000 7f140fa72700 1 -- 172.21.15.98:0/2300092659 wait complete. 2023-10-06T01:40:28.650 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:28.659 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.773+0000 7fb46c310700 1 Processor -- start 2023-10-06T01:40:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.773+0000 7fb46c310700 1 -- start start 2023-10-06T01:40:28.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46c310700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb4640b07a0 0x7fb4640ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46c310700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b0ca0 con 0x7fb4641595a0 2023-10-06T01:40:28.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46c310700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b0f80 con 0x7fb4640b49e0 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46c310700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640ab390 con 0x7fb4640b07a0 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46a8ad700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb4640b07a0 0x7fb4640ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46a0ac700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb4641595a0 0x7fb4640aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41800/0 (socket says 172.21.15.98:41800) 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.774+0000 7fb46a0ac700 1 -- 172.21.15.98:0/2552257644 learned_addr learned my addr 172.21.15.98:0/2552257644 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46a8ad700 1 -- 172.21.15.98:0/2552257644 >> v1:172.21.15.184:6789/0 conn(0x7fb4640b49e0 legacy=0x7fb4640aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46a8ad700 1 -- 172.21.15.98:0/2552257644 >> v1:172.21.15.98:6789/0 conn(0x7fb4641595a0 legacy=0x7fb4640aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46a8ad700 1 -- 172.21.15.98:0/2552257644 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640b1260 con 0x7fb4640b07a0 2023-10-06T01:40:28.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb4690aa700 1 -- 172.21.15.98:0/2552257644 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263504731 0 0) 0x7fb4640b0ca0 con 0x7fb4641595a0 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46a8ad700 1 --2- 172.21.15.98:0/2552257644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb4640ccfd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e90 tx=0x7fb45c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10acaf3caf96efbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb4690aa700 1 -- 172.21.15.98:0/2552257644 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00e070 con 0x7fb4640b07a0 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb4690aa700 1 -- 172.21.15.98:0/2552257644 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00bbb0 con 0x7fb4640b07a0 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb4690aa700 1 -- 172.21.15.98:0/2552257644 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c012430 con 0x7fb4640b07a0 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46c310700 1 -- 172.21.15.98:0/2552257644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 msgr2=0x7fb4640ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46c310700 1 --2- 172.21.15.98:0/2552257644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb4640ccfd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e90 tx=0x7fb45c009780 comp rx=0 tx=0).stop 2023-10-06T01:40:28.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.775+0000 7fb46c310700 1 -- 172.21.15.98:0/2552257644 shutdown_connections 2023-10-06T01:40:28.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.776+0000 7fb46c310700 1 --2- 172.21.15.98:0/2552257644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb4640ccfd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.776+0000 7fb46c310700 1 -- 172.21.15.98:0/2552257644 >> 172.21.15.98:0/2552257644 conn(0x7fb464099830 msgr2=0x7fb46406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.776+0000 7fb46c310700 1 -- 172.21.15.98:0/2552257644 shutdown_connections 2023-10-06T01:40:28.779 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.776+0000 7fb46c310700 1 -- 172.21.15.98:0/2552257644 wait complete. 2023-10-06T01:40:28.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 Processor -- start 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 -- start start 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b49e0 0x7fb46409a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4641595a0 0x7fb46415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464159a10 con 0x7fb4640b07a0 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464159cf0 con 0x7fb4641595a0 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46c310700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464159fd0 con 0x7fb4640b49e0 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46a0ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46a0ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54130/0 (socket says 172.21.15.98:54130) 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.777+0000 7fb46a0ac700 1 -- 172.21.15.98:0/2378763070 learned_addr learned my addr 172.21.15.98:0/2378763070 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:28.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb4698ab700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b49e0 0x7fb46409a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a8ad700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4641595a0 0x7fb46415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 -- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b49e0 msgr2=0x7fb46409a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b49e0 0x7fb46409a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 -- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4641595a0 msgr2=0x7fb46415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4641595a0 0x7fb46415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 -- 172.21.15.98:0/2378763070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c009a20 con 0x7fb4640b07a0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46a0ac700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c5e0 tx=0x7fb460007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600072e0 con 0x7fb4640b07a0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460007460 con 0x7fb4640b07a0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46415a2b0 con 0x7fb4640b07a0 2023-10-06T01:40:28.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600057f0 con 0x7fb4640b07a0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.778+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464158ea0 con 0x7fb4640b07a0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.779+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464064ad0 con 0x7fb4640b07a0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.779+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb460005c00 con 0x7fb4640b07a0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.780+0000 7fb45a7fc700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 0x7fb4480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.780+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb460083e60 con 0x7fb4640b07a0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.780+0000 7fb4698ab700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 0x7fb4480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.780+0000 7fb4698ab700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 0x7fb4480693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4640b62c0 tx=0x7fb454007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:28.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.784+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb460050c80 con 0x7fb4640b07a0 2023-10-06T01:40:28.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.969+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4640b1540 con 0x7fb4640b07a0 2023-10-06T01:40:28.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.970+0000 7fb45a7fc700 1 -- 172.21.15.98:0/2378763070 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fb460017020 con 0x7fb4640b07a0 2023-10-06T01:40:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:28.971 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 msgr2=0x7fb4480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 0x7fb4480693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4640b62c0 tx=0x7fb454007ec0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 msgr2=0x7fb464158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c5e0 tx=0x7fb460007e80 comp rx=0 tx=0).stop 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 shutdown_connections 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb448066f30 0x7fb4480693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4641595a0 0x7fb46415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4640b07a0 0x7fb464158960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 --2- 172.21.15.98:0/2378763070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4640b49e0 0x7fb46409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:28.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 >> 172.21.15.98:0/2378763070 conn(0x7fb464099830 msgr2=0x7fb464069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:28.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 shutdown_connections 2023-10-06T01:40:28.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:28.972+0000 7fb46c310700 1 -- 172.21.15.98:0/2378763070 wait complete. 2023-10-06T01:40:28.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T01:40:28.985 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:29.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.098+0000 7f79562d2700 1 Processor -- start 2023-10-06T01:40:29.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.098+0000 7f79562d2700 1 -- start start 2023-10-06T01:40:29.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79501595a0 0x7f79500aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 0x7f79500aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b07a0 0x7f79500b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500b0ca0 con 0x7f79501595a0 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500b0f80 con 0x7f79500b49e0 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f79562d2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79500b1260 con 0x7f79500b07a0 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f794ffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79501595a0 0x7f79500aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f794ffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79501595a0 0x7f79500aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54144/0 (socket says 172.21.15.98:54144) 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.099+0000 7f794ffff700 1 -- 172.21.15.98:0/2625746089 learned_addr learned my addr 172.21.15.98:0/2625746089 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f795486f700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b07a0 0x7f79500b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794f7fe700 1 --2- 172.21.15.98:0/2625746089 >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 0x7f79500aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 -- 172.21.15.98:0/2625746089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b07a0 msgr2=0x7f79500b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b07a0 0x7f79500b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 -- 172.21.15.98:0/2625746089 >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 msgr2=0x7f79500aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 --2- 172.21.15.98:0/2625746089 >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 0x7f79500aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 -- 172.21.15.98:0/2625746089 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500b1540 con 0x7f79501595a0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794f7fe700 1 --2- 172.21.15.98:0/2625746089 >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 0x7f79500aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794ffff700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79500aa570 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7940009990 tx=0x7f7940009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a6d2a6940774e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794effd700 1 -- 172.21.15.98:0/2625746089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940009e80 con 0x7f79501595a0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794effd700 1 -- 172.21.15.98:0/2625746089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7940005c10 con 0x7f79501595a0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.100+0000 7f794effd700 1 -- 172.21.15.98:0/2625746089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794000b610 con 0x7f79501595a0 2023-10-06T01:40:29.103 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 -- 172.21.15.98:0/2625746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 msgr2=0x7f79500aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79500aa570 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7940009990 tx=0x7f7940009220 comp rx=0 tx=0).stop 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 -- 172.21.15.98:0/2625746089 shutdown_connections 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b07a0 0x7f79500b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 --2- 172.21.15.98:0/2625746089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79500aa570 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 --2- 172.21.15.98:0/2625746089 >> v2:172.21.15.184:3300/0 conn(0x7f79500b49e0 0x7f79500aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 -- 172.21.15.98:0/2625746089 >> 172.21.15.98:0/2625746089 conn(0x7f7950099830 msgr2=0x7f795006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:29.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 -- 172.21.15.98:0/2625746089 shutdown_connections 2023-10-06T01:40:29.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.101+0000 7f79562d2700 1 -- 172.21.15.98:0/2625746089 wait complete. 2023-10-06T01:40:29.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.102+0000 7f79562d2700 1 Processor -- start 2023-10-06T01:40:29.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.102+0000 7f79562d2700 1 -- start start 2023-10-06T01:40:29.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500b07a0 0x7f79501539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b49e0 0x7f7950169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015ac70 con 0x7f79501595a0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015af10 con 0x7f79500b07a0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795015b1f0 con 0x7f79500b49e0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54156/0 (socket says 172.21.15.98:54156) 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 -- 172.21.15.98:0/3553366975 learned_addr learned my addr 172.21.15.98:0/3553366975 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f794ffff700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500b07a0 0x7f79501539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f794f7fe700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b49e0 0x7f7950169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 -- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b49e0 msgr2=0x7f7950169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b49e0 0x7f7950169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 -- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500b07a0 msgr2=0x7f79501539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500b07a0 0x7f79501539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 -- 172.21.15.98:0/3553366975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940009480 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f795486f700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7948009ef0 tx=0x7f794800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794800cc50 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794800f040 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795015b4d0 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948014480 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.103+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795016a120 con 0x7f79501595a0 2023-10-06T01:40:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.104+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7950064ad0 con 0x7f79501595a0 2023-10-06T01:40:29.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.105+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7948014830 con 0x7f79501595a0 2023-10-06T01:40:29.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.105+0000 7f7937fff700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 0x7f793c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.105+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7948085720 con 0x7f79501595a0 2023-10-06T01:40:29.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.105+0000 7f794ffff700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 0x7f793c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.106+0000 7f794ffff700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 0x7f793c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7940009260 tx=0x7f7940016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.111 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.109+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7948052540 con 0x7f79501595a0 2023-10-06T01:40:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.334+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79500b1820 con 0x7f79501595a0 2023-10-06T01:40:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.335+0000 7f7937fff700 1 -- 172.21.15.98:0/3553366975 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f7948055b60 con 0x7f79501595a0 2023-10-06T01:40:29.335 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:29.336 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 msgr2=0x7f793c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 0x7f793c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7940009260 tx=0x7f7940016000 comp rx=0 tx=0).stop 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 msgr2=0x7f79501518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7948009ef0 tx=0x7f794800b040 comp rx=0 tx=0).stop 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 shutdown_connections 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f793c066f30 0x7f793c0693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79501595a0 0x7f79501518a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79500b07a0 0x7f79501539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 --2- 172.21.15.98:0/3553366975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79500b49e0 0x7f7950169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 >> 172.21.15.98:0/3553366975 conn(0x7f7950099830 msgr2=0x7f7950069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:29.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 shutdown_connections 2023-10-06T01:40:29.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.337+0000 7f79562d2700 1 -- 172.21.15.98:0/3553366975 wait complete. 2023-10-06T01:40:29.339 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:29.350 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.464+0000 7f8c6897f700 1 Processor -- start 2023-10-06T01:40:29.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.464+0000 7f8c6897f700 1 -- start start 2023-10-06T01:40:29.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6897f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8c641595a0 0x7f8c640aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6897f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c640aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6897f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640b0ca0 con 0x7f8c640b07a0 2023-10-06T01:40:29.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6897f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c640b0f80 con 0x7f8c641595a0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6897f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640b1260 con 0x7f8c640b49e0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c62d9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8c640b07a0 0x7f8c640b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41810/0 (socket says 172.21.15.98:41810) 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4032257068 learned_addr learned my addr 172.21.15.98:0/4032257068 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725950655 0 0) 0x7f8c640b0ca0 con 0x7f8c640b07a0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6259c700 1 --2- 172.21.15.98:0/4032257068 >> v2:172.21.15.184:3300/0 conn(0x7f8c641595a0 0x7f8c640aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c4c003570 con 0x7f8c640b07a0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c61d9b700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c640aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199301179 0 0) 0x7f8c4c003570 con 0x7f8c640b07a0 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 msgr2=0x7f8c640aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c640aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 >> v2:172.21.15.184:3300/0 conn(0x7f8c641595a0 msgr2=0x7f8c640aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 --2- 172.21.15.98:0/4032257068 >> v2:172.21.15.184:3300/0 conn(0x7f8c641595a0 0x7f8c640aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640b1540 con 0x7f8c640b07a0 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f8c5c0017f0 con 0x7f8c640b07a0 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 >> v1:172.21.15.98:6789/0 conn(0x7f8c640b07a0 legacy=0x7f8c640b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c003fe0 0x7f8c4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c4c004d50 0x7f8c4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640b0f80 con 0x7f8c4c004d50 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640b1260 con 0x7f8c4c003fe0 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c00ba90 con 0x7f8c640b49e0 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.465+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c4c004d50 0x7f8c4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c6259c700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c61d9b700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c003fe0 0x7f8c4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 msgr2=0x7f8c4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640b49e0 0x7f8c4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c003fe0 msgr2=0x7f8c4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c003fe0 0x7f8c4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4032257068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c003570 con 0x7f8c4c004d50 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c4c004d50 0x7f8c4c0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00a970 tx=0x7f8c5c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9d01c8d34fd0dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c012440 con 0x7f8c4c004d50 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0125c0 con 0x7f8c4c004d50 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.466+0000 7f8c6159a700 1 -- 172.21.15.98:0/4032257068 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c012980 con 0x7f8c4c004d50 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 -- 172.21.15.98:0/4032257068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c4c004d50 msgr2=0x7f8c4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4032257068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c4c004d50 0x7f8c4c0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00a970 tx=0x7f8c5c002210 comp rx=0 tx=0).stop 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4032257068 reap_dead start 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 -- 172.21.15.98:0/4032257068 shutdown_connections 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 -- 172.21.15.98:0/4032257068 >> 172.21.15.98:0/4032257068 conn(0x7f8c64099830 msgr2=0x7f8c6406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 -- 172.21.15.98:0/4032257068 shutdown_connections 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.467+0000 7f8c6897f700 1 -- 172.21.15.98:0/4032257068 wait complete. 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 Processor -- start 2023-10-06T01:40:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 -- start start 2023-10-06T01:40:29.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c004d50 0x7f8c6409cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 0x7f8c6416a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.474 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6409d430 con 0x7f8c640ce990 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.468+0000 7f8c6897f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640cf370 con 0x7f8c4c004d50 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c6897f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6416d700 con 0x7f8c6416d200 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54166/0 (socket says 172.21.15.98:54166) 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4283150612 learned_addr learned my addr 172.21.15.98:0/4283150612 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c61d9b700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 0x7f8c6416a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c6259c700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c004d50 0x7f8c6409cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 msgr2=0x7f8c6416a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 0x7f8c6416a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c004d50 msgr2=0x7f8c6409cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c004d50 0x7f8c6409cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 -- 172.21.15.98:0/4283150612 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6416d9e0 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c61d9b700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 0x7f8c6416a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c62d9d700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c000f80 tx=0x7f8c5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c023070 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c012440 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6416dcc0 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0017b0 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.469+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c641572b0 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.470+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c64064ad0 con 0x7f8c640ce990 2023-10-06T01:40:29.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.470+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c5c019030 con 0x7f8c640ce990 2023-10-06T01:40:29.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.471+0000 7f8c527fc700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 0x7f8c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.471+0000 7f8c6259c700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 0x7f8c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.471+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8c5c085f70 con 0x7f8c640ce990 2023-10-06T01:40:29.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.471+0000 7f8c6259c700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 0x7f8c440693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006e70 tx=0x7f8c54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.475+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c5c052d90 con 0x7f8c640ce990 2023-10-06T01:40:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.699+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c640b1820 con 0x7f8c640ce990 2023-10-06T01:40:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.699+0000 7f8c527fc700 1 -- 172.21.15.98:0/4283150612 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f8c5c017070 con 0x7f8c640ce990 2023-10-06T01:40:29.700 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:29.700 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/761820022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":761820022},{"type":"v1","addr":"172.21.15.98:6836","nonce":761820022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3955349002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3955349002},{"type":"v1","addr":"172.21.15.184:6835","nonce":3955349002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1132951311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1132951311},{"type":"v1","addr":"172.21.15.184:6833","nonce":1132951311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:39:44.696941+0000","modified":"2023-10-06T01:39:58.009938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1024246488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1024246488},{"type":"v1","addr":"172.21.15.98:6837","nonce":1024246488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T01:40:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 msgr2=0x7f8c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 0x7f8c440693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006e70 tx=0x7f8c54008040 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 msgr2=0x7f8c640cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c000f80 tx=0x7f8c5c015000 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 shutdown_connections 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c44066f30 0x7f8c440693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c640ce990 0x7f8c640cee30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c4c004d50 0x7f8c6409cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 --2- 172.21.15.98:0/4283150612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c6416d200 0x7f8c6416a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 >> 172.21.15.98:0/4283150612 conn(0x7f8c64099830 msgr2=0x7f8c64158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 shutdown_connections 2023-10-06T01:40:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.702+0000 7f8c6897f700 1 -- 172.21.15.98:0/4283150612 wait complete. 2023-10-06T01:40:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2023-10-06T01:40:29.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.828+0000 7f3051ae6700 1 Processor -- start 2023-10-06T01:40:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.828+0000 7f3051ae6700 1 -- start start 2023-10-06T01:40:29.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f3051ae6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 0x7f304c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f3051ae6700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f304c159870 0x7f304c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f3051ae6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c0cbfe0 con 0x7f304c0b29f0 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f3051ae6700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c0cb3d0 con 0x7f304c159870 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f3051ae6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0cb6b0 con 0x7f304c0b2620 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.829+0000 7f304bfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f304c159870 0x7f304c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f304c159870 0x7f304c0cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54168/0 (socket says 172.21.15.98:54168) 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 -- 172.21.15.98:0/1414841373 learned_addr learned my addr 172.21.15.98:0/1414841373 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304b7fe700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 0x7f304c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 -- 172.21.15.98:0/1414841373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 msgr2=0x7f304c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 0x7f304c0b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 -- 172.21.15.98:0/1414841373 >> v1:172.21.15.184:6789/0 conn(0x7f304c0b29f0 legacy=0x7f304c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 -- 172.21.15.98:0/1414841373 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0cb990 con 0x7f304c159870 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304b7fe700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 0x7f304c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304bfff700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c159870 0x7f304c0cc470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30440099b0 tx=0x7f3044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b134f52b52dcb03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304a7fc700 1 -- 172.21.15.98:0/1414841373 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400b860 con 0x7f304c159870 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304a7fc700 1 -- 172.21.15.98:0/1414841373 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304400b9c0 con 0x7f304c159870 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.830+0000 7f304a7fc700 1 -- 172.21.15.98:0/1414841373 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400bd60 con 0x7f304c159870 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 -- 172.21.15.98:0/1414841373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c159870 msgr2=0x7f304c0cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c159870 0x7f304c0cc470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30440099b0 tx=0x7f3044009250 comp rx=0 tx=0).stop 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 -- 172.21.15.98:0/1414841373 shutdown_connections 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c159870 0x7f304c0cc470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1414841373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c0b2620 0x7f304c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 -- 172.21.15.98:0/1414841373 >> 172.21.15.98:0/1414841373 conn(0x7f304c06ed20 msgr2=0x7f304c06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 -- 172.21.15.98:0/1414841373 shutdown_connections 2023-10-06T01:40:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.831+0000 7f3051ae6700 1 -- 172.21.15.98:0/1414841373 wait complete. 2023-10-06T01:40:29.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.832+0000 7f3051ae6700 1 Processor -- start 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.832+0000 7f3051ae6700 1 -- start start 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f304c0b2620 0x7f304c14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c159870 0x7f304c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14c500 con 0x7f304c0b29f0 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14b8f0 con 0x7f304c0b2620 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14bbd0 con 0x7f304c159870 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54180/0 (socket says 172.21.15.98:54180) 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 -- 172.21.15.98:0/1980095189 learned_addr learned my addr 172.21.15.98:0/1980095189 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304b7fe700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f304c0b2620 0x7f304c14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304bfff700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c159870 0x7f304c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 -- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c159870 msgr2=0x7f304c14c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c159870 0x7f304c14c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 -- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f304c0b2620 msgr2=0x7f304c14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f304c0b2620 0x7f304c14aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 -- 172.21.15.98:0/1980095189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440094f0 con 0x7f304c0b29f0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f304affd700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f304000c590 tx=0x7f304000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040011590 con 0x7f304c0b29f0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040011710 con 0x7f304c0b29f0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c14beb0 con 0x7f304c0b29f0 2023-10-06T01:40:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040011ad0 con 0x7f304c0b29f0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.833+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c14af90 con 0x7f304c0b29f0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.834+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c064ad0 con 0x7f304c0b29f0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.835+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f304001b3b0 con 0x7f304c0b29f0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.835+0000 7f30337fe700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 0x7f30380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.835+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3040083e10 con 0x7f304c0b29f0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.835+0000 7f304b7fe700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 0x7f30380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.835+0000 7f304b7fe700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 0x7f30380693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f303c0077a0 tx=0x7f303c009a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:29.839+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3040050c30 con 0x7f304c0b29f0 2023-10-06T01:40:30.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.023+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f304c0cbc70 con 0x7f304c0b29f0 2023-10-06T01:40:30.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.024+0000 7f30337fe700 1 -- 172.21.15.98:0/1980095189 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f3040017070 con 0x7f304c0b29f0 2023-10-06T01:40:30.025 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:30.025 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:39:49.732140+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:39:42.361455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T01:39:43.656625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T01:39:47.351521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:30.026 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.025+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 msgr2=0x7f30380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.025+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 0x7f30380693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f303c0077a0 tx=0x7f303c009a20 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.025+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 msgr2=0x7f304c0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f304000c590 tx=0x7f304000c8a0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 shutdown_connections 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3038066f30 0x7f30380693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f304c159870 0x7f304c14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f304c0b2620 0x7f304c14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 --2- 172.21.15.98:0/1980095189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c0b29f0 0x7f304c0c5990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.027 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 >> 172.21.15.98:0/1980095189 conn(0x7f304c06ed20 msgr2=0x7f304c0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:30.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 shutdown_connections 2023-10-06T01:40:30.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.026+0000 7f3051ae6700 1 -- 172.21.15.98:0/1980095189 wait complete. 2023-10-06T01:40:30.039 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T01:40:30.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.153+0000 7f9cd6ce6700 1 Processor -- start 2023-10-06T01:40:30.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.153+0000 7f9cd6ce6700 1 -- start start 2023-10-06T01:40:30.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd6ce6700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9cd00ca3b0 0x7f9cd01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd6ce6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00adba0 0x7f9cd00ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd6ce6700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00ceb70 con 0x7f9cd00ce710 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd6ce6700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00cbbd0 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd6ce6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0157e20 con 0x7f9cd00adba0 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd4a82700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9cd00ca3b0 0x7f9cd01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd5283700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9cd00ce710 0x7f9cd015af10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41812/0 (socket says 172.21.15.98:41812) 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd4a82700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9cd00ca3b0 0x7f9cd01582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54182/0 (socket says 172.21.15.98:54182) 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cd5283700 1 -- 172.21.15.98:0/294921911 learned_addr learned my addr 172.21.15.98:0/294921911 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.154+0000 7f9cc7fff700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00adba0 0x7f9cd00ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd4a82700 1 -- 172.21.15.98:0/294921911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00adba0 msgr2=0x7f9cd00ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd4a82700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00adba0 0x7f9cd00ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd4a82700 1 -- 172.21.15.98:0/294921911 >> v1:172.21.15.98:6789/0 conn(0x7f9cd00ce710 legacy=0x7f9cd015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd4a82700 1 -- 172.21.15.98:0/294921911 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00cb610 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cc77fe700 1 -- 172.21.15.98:0/294921911 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310535823 0 0) 0x7f9cc8001110 con 0x7f9cd00ce710 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd4a82700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd01582b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009e60 tx=0x7f9ccc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e212e6af85863743 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cc77fe700 1 -- 172.21.15.98:0/294921911 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00e070 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cc77fe700 1 -- 172.21.15.98:0/294921911 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc00bb60 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cc77fe700 1 -- 172.21.15.98:0/294921911 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc012430 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/294921911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00ca3b0 msgr2=0x7f9cd01582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd01582b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009e60 tx=0x7f9ccc009780 comp rx=0 tx=0).stop 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/294921911 shutdown_connections 2023-10-06T01:40:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd01582b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/294921911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00adba0 0x7f9cd00ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/294921911 >> 172.21.15.98:0/294921911 conn(0x7f9cd0070810 msgr2=0x7f9cd0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:30.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/294921911 shutdown_connections 2023-10-06T01:40:30.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.155+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/294921911 wait complete. 2023-10-06T01:40:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.156+0000 7f9cd6ce6700 1 Processor -- start 2023-10-06T01:40:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 -- start start 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd0068f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00ce710 0x7f9cd00c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd006a410 con 0x7f9cd00adba0 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0069800 con 0x7f9cd00ca3b0 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd6ce6700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0069ae0 con 0x7f9cd00ce710 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.157+0000 7f9cd4a82700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54192/0 (socket says 172.21.15.98:54192) 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 -- 172.21.15.98:0/2206489397 learned_addr learned my addr 172.21.15.98:0/2206489397 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:30.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd5283700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00ce710 0x7f9cd00c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cc7fff700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd0068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 -- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00ce710 msgr2=0x7f9cd00c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00ce710 0x7f9cd00c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 -- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd00ca3b0 msgr2=0x7f9cd0068f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd0068f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 -- 172.21.15.98:0/2206489397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc009a20 con 0x7f9cd00adba0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd4a82700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00cdfe0 tx=0x7f9ccc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc012c80 con 0x7f9cd00adba0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc012e00 con 0x7f9cd00adba0 2023-10-06T01:40:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0276c0 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd0069dc0 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.158+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd014dd40 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.159+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd0064ad0 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.159+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ccc024070 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.160+0000 7f9cc4ff9700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.160+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9ccc086df0 con 0x7f9cd00adba0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.160+0000 7f9cc7fff700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.160+0000 7f9cc7fff700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 0x7f9cbc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0151790 tx=0x7f9cc00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.164+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ccc053c10 con 0x7f9cd00adba0 2023-10-06T01:40:30.389 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.389+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9cd00cb8f0 con 0x7f9cd00adba0 2023-10-06T01:40:30.390 INFO:tasks.ceph.mon.a.smithi098.stderr:2023-10-06T01:40:30.389+0000 7f8a6565e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T01:40:30.398 INFO:tasks.ceph.mds.c.smithi098.stderr: -3579> 2023-10-06T01:39:50.412+0000 7f03b850f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:40:30.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.398+0000 7f9cc4ff9700 1 -- 172.21.15.98:0/2206489397 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9ccc02c020 con 0x7f9cd00adba0 2023-10-06T01:40:30.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 msgr2=0x7f9cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 0x7f9cbc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0151790 tx=0x7f9cc00091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 msgr2=0x7f9cd014d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00cdfe0 tx=0x7f9ccc019000 comp rx=0 tx=0).stop 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 shutdown_connections 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cd00ce710 0x7f9cd00c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00adba0 0x7f9cd014d800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 --2- 172.21.15.98:0/2206489397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cd00ca3b0 0x7f9cd0068f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 >> 172.21.15.98:0/2206489397 conn(0x7f9cd0070810 msgr2=0x7f9cd00cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 shutdown_connections 2023-10-06T01:40:30.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.400+0000 7f9cd6ce6700 1 -- 172.21.15.98:0/2206489397 wait complete. 2023-10-06T01:40:30.403 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T01:40:30.414 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T01:40:30.423 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2023-10-06T01:40:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.531+0000 7fca0983b700 1 Processor -- start 2023-10-06T01:40:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.531+0000 7fca0983b700 1 -- start start 2023-10-06T01:40:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca0983b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca04158420 0x7fca04162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca0983b700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca04159ab0 con 0x7fca0406bff0 2023-10-06T01:40:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca0983b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040afbf0 con 0x7fca0415b980 2023-10-06T01:40:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca0983b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca04158880 con 0x7fca04158420 2023-10-06T01:40:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca027fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fca0406bff0 0x7fca04157d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41820/0 (socket says 172.21.15.98:41820) 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca027fc700 1 -- 172.21.15.98:0/2262818977 learned_addr learned my addr 172.21.15.98:0/2262818977 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> v2:172.21.15.184:3300/0 conn(0x7fca04158420 0x7fca04162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146082507 0 0) 0x7fca04159ab0 con 0x7fca0406bff0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9f0003570 con 0x7fca0406bff0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676385958 0 0) 0x7fc9f0003570 con 0x7fca0406bff0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 >> v2:172.21.15.184:3300/0 conn(0x7fca04158420 msgr2=0x7fca04162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 --2- 172.21.15.98:0/2262818977 >> v2:172.21.15.184:3300/0 conn(0x7fca04158420 0x7fca04162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 >> v1:172.21.15.184:6789/0 conn(0x7fca0415b980 legacy=0x7fca0406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040afed0 con 0x7fca0406bff0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.532+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fc9ec001470 con 0x7fca0406bff0 2023-10-06T01:40:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 >> v1:172.21.15.98:6789/0 conn(0x7fca0406bff0 legacy=0x7fca04157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f0003a60 0x7fc9f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9f0004810 0x7fc9f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 0x7fc9f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04158880 con 0x7fc9f00052e0 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f000c2a0 con 0x7fc9f0003a60 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f000c460 con 0x7fc9f0004810 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> v2:172.21.15.184:3300/0 conn(0x7fca04158420 0x7fca04162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 0x7fc9f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca027fc700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9f0004810 0x7fc9f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca02ffd700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f0003a60 0x7fc9f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 -- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9f0004810 msgr2=0x7fc9f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9f0004810 0x7fc9f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 -- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f0003a60 msgr2=0x7fc9f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f0003a60 0x7fc9f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 -- 172.21.15.98:0/2262818977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0003570 con 0x7fc9f00052e0 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 -- 172.21.15.98:0/2262818977 reap_dead start 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca037fe700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 0x7fc9f000ba60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400e3c0 tx=0x7fc9f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d07cec2e310f364 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4009c50 con 0x7fc9f00052e0 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400f040 con 0x7fc9f00052e0 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.533+0000 7fca01ffb700 1 -- 172.21.15.98:0/2262818977 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400eea0 con 0x7fc9f00052e0 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 -- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 msgr2=0x7fc9f000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 0x7fc9f000ba60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400e3c0 tx=0x7fc9f400b040 comp rx=0 tx=0).stop 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 -- 172.21.15.98:0/2262818977 shutdown_connections 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 --2- 172.21.15.98:0/2262818977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9f00052e0 0x7fc9f000ba60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 -- 172.21.15.98:0/2262818977 >> 172.21.15.98:0/2262818977 conn(0x7fca040ce330 msgr2=0x7fca040ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 -- 172.21.15.98:0/2262818977 shutdown_connections 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.534+0000 7fca0983b700 1 -- 172.21.15.98:0/2262818977 wait complete. 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.535+0000 7fca0983b700 1 Processor -- start 2023-10-06T01:40:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.535+0000 7fca0983b700 1 -- start start 2023-10-06T01:40:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f00052e0 0x7fca040b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca040b10e0 0x7fca0414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040c13b0 con 0x7fca04069f20 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040c9860 con 0x7fc9f00052e0 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca0983b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040ca6c0 con 0x7fca040b10e0 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54230/0 (socket says 172.21.15.98:54230) 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 -- 172.21.15.98:0/3498395304 learned_addr learned my addr 172.21.15.98:0/3498395304 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca02ffd700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f00052e0 0x7fca040b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca037fe700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca040b10e0 0x7fca0414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 -- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca040b10e0 msgr2=0x7fca0414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca040b10e0 0x7fca0414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 -- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f00052e0 msgr2=0x7fca040b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f00052e0 0x7fca040b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 -- 172.21.15.98:0/3498395304 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040c9ab0 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.536+0000 7fca027fc700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec004b90 tx=0x7fc9ec016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec019040 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec007760 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec01b510 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca040c9d90 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040ca350 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.537+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca040aebc0 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.538+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9ec01b940 con 0x7fca04069f20 2023-10-06T01:40:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.538+0000 7fc9faffd700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.538+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fc9ec08c940 con 0x7fca04069f20 2023-10-06T01:40:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.538+0000 7fca02ffd700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.539+0000 7fca02ffd700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 0x7fc9e40693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00e460 tx=0x7fc9fc010040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.542+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9ec059710 con 0x7fca04069f20 2023-10-06T01:40:30.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:30.765+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca04064ad0 con 0x7fca04069f20 2023-10-06T01:40:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.397+0000 7fc9faffd700 1 -- 172.21.15.98:0/3498395304 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9ec023020 con 0x7fca04069f20 2023-10-06T01:40:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 msgr2=0x7fc9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 0x7fc9e40693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00e460 tx=0x7fc9fc010040 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 msgr2=0x7fca040c0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec004b90 tx=0x7fc9ec016040 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 shutdown_connections 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca040b10e0 0x7fca0414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9f00052e0 0x7fca040b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 --2- 172.21.15.98:0/3498395304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca04069f20 0x7fca040c0e70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 >> 172.21.15.98:0/3498395304 conn(0x7fca040ce330 msgr2=0x7fca04068700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 shutdown_connections 2023-10-06T01:40:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.400+0000 7fca0983b700 1 -- 172.21.15.98:0/3498395304 wait complete. 2023-10-06T01:40:31.413 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T01:40:31.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.572+0000 7fce585f7700 1 Processor -- start 2023-10-06T01:40:31.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.572+0000 7fce585f7700 1 -- start start 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fce5006ed20 0x7fce500b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006f0f0 0x7fce500b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fce500aee20 0x7fce500af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5006fb80 con 0x7fce5006ed20 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce500b2d20 con 0x7fce500aee20 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce585f7700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b0450 con 0x7fce5006f0f0 2023-10-06T01:40:31.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fce5006ed20 0x7fce500b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fce5006ed20 0x7fce500b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54246/0 (socket says 172.21.15.98:54246) 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 -- 172.21.15.98:0/10843195 learned_addr learned my addr 172.21.15.98:0/10843195 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 -- 172.21.15.98:0/10843195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006f0f0 msgr2=0x7fce500b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 --2- 172.21.15.98:0/10843195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006f0f0 0x7fce500b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 -- 172.21.15.98:0/10843195 >> v2:172.21.15.184:3300/0 conn(0x7fce500aee20 msgr2=0x7fce500af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 --2- 172.21.15.98:0/10843195 >> v2:172.21.15.184:3300/0 conn(0x7fce500aee20 0x7fce500af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 -- 172.21.15.98:0/10843195 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500af7c0 con 0x7fce5006ed20 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce56393700 1 --2- 172.21.15.98:0/10843195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006ed20 0x7fce500b23a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1390 tx=0x7fce4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd6d58e83650c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:31.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce55391700 1 -- 172.21.15.98:0/10843195 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00b860 con 0x7fce5006ed20 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce55391700 1 -- 172.21.15.98:0/10843195 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00b9c0 con 0x7fce5006ed20 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.573+0000 7fce55391700 1 -- 172.21.15.98:0/10843195 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00bd60 con 0x7fce5006ed20 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 -- 172.21.15.98:0/10843195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006ed20 msgr2=0x7fce500b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 --2- 172.21.15.98:0/10843195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006ed20 0x7fce500b23a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1390 tx=0x7fce4c009250 comp rx=0 tx=0).stop 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 -- 172.21.15.98:0/10843195 shutdown_connections 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 --2- 172.21.15.98:0/10843195 >> v2:172.21.15.184:3300/0 conn(0x7fce500aee20 0x7fce500af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 --2- 172.21.15.98:0/10843195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006ed20 0x7fce500b23a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 --2- 172.21.15.98:0/10843195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006f0f0 0x7fce500b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 -- 172.21.15.98:0/10843195 >> 172.21.15.98:0/10843195 conn(0x7fce50068330 msgr2=0x7fce50068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:31.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 -- 172.21.15.98:0/10843195 shutdown_connections 2023-10-06T01:40:31.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.574+0000 7fce585f7700 1 -- 172.21.15.98:0/10843195 wait complete. 2023-10-06T01:40:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.575+0000 7fce585f7700 1 Processor -- start 2023-10-06T01:40:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.575+0000 7fce585f7700 1 -- start start 2023-10-06T01:40:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006ed20 0x7fce500a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce500aee20 0x7fce50168a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500a7660 con 0x7fce5006f0f0 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500a7940 con 0x7fce500aee20 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce585f7700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500a7c20 con 0x7fce5006ed20 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54258/0 (socket says 172.21.15.98:54258) 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 -- 172.21.15.98:0/157994600 learned_addr learned my addr 172.21.15.98:0/157994600 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce56393700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006ed20 0x7fce500a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce56b94700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce500aee20 0x7fce50168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 -- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006ed20 msgr2=0x7fce500a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:31.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006ed20 0x7fce500a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 -- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce500aee20 msgr2=0x7fce50168a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce500aee20 0x7fce50168a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 -- 172.21.15.98:0/157994600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0094f0 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.576+0000 7fce55b92700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fce4000b810 tx=0x7fce40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.577+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000f510 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.577+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4000f690 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.577+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500a7f00 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.577+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000fa50 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.577+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce500a7360 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.578+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce50064ad0 con 0x7fce5006f0f0 2023-10-06T01:40:31.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.578+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce4001a400 con 0x7fce5006f0f0 2023-10-06T01:40:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.578+0000 7fce467fc700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.578+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fce40084500 con 0x7fce5006f0f0 2023-10-06T01:40:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.578+0000 7fce56393700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.579+0000 7fce56393700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 0x7fce340693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1390 tx=0x7fce4c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.583+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce40051350 con 0x7fce5006f0f0 2023-10-06T01:40:31.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:31.773+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce500b2a40 con 0x7fce5006f0f0 2023-10-06T01:40:32.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.400+0000 7fce467fc700 1 -- 172.21.15.98:0/157994600 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce40054970 con 0x7fce5006f0f0 2023-10-06T01:40:32.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.405+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 msgr2=0x7fce340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.405+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 0x7fce340693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1390 tx=0x7fce4c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.405+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 msgr2=0x7fce500ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.405+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fce4000b810 tx=0x7fce40009250 comp rx=0 tx=0).stop 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 shutdown_connections 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce500aee20 0x7fce50168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce5006ed20 0x7fce500a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 --2- 172.21.15.98:0/157994600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce5006f0f0 0x7fce500ab8c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.406+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 >> 172.21.15.98:0/157994600 conn(0x7fce50068330 msgr2=0x7fce500cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:32.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.408+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 shutdown_connections 2023-10-06T01:40:32.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.408+0000 7fce585f7700 1 -- 172.21.15.98:0/157994600 wait complete. 2023-10-06T01:40:32.415 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2023-10-06T01:40:32.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T01:40:32.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.546+0000 7f9ecdbd5700 1 Processor -- start 2023-10-06T01:40:32.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.546+0000 7f9ecdbd5700 1 -- start start 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ecdbd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec8067b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ecdbd5700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ec8159020 0x7f9ec81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ecdbd5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80cced0 con 0x7f9ec815a8a0 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ecdbd5700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec815a410 con 0x7f9ec8159020 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ecdbd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80ccbf0 con 0x7f9ec80677b0 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ec7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ec8159020 0x7f9ec81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ec77fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec8067b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ec7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ec8159020 0x7f9ec81594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:41500/0 (socket says 172.21.15.98:41500) 2023-10-06T01:40:32.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.547+0000 7f9ec7fff700 1 -- 172.21.15.98:0/3667500320 learned_addr learned my addr 172.21.15.98:0/3667500320 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:32.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec7fff700 1 -- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 msgr2=0x7f9ec8067b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:32.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec7fff700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec8067b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec7fff700 1 -- 172.21.15.98:0/3667500320 >> v1:172.21.15.184:6789/0 conn(0x7f9ec815a8a0 legacy=0x7f9ec80cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec67fc700 1 -- 172.21.15.98:0/3667500320 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518560456 0 0) 0x7f9ec80cced0 con 0x7f9ec815a8a0 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec7fff700 1 -- 172.21.15.98:0/3667500320 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8159dc0 con 0x7f9ec8159020 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec77fe700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec8067b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec7fff700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec8159020 0x7f9ec81594a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009f20 tx=0x7f9ec0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9712f19cdd1f94e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec67fc700 1 -- 172.21.15.98:0/3667500320 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000e070 con 0x7f9ec8159020 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec67fc700 1 -- 172.21.15.98:0/3667500320 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000b9d0 con 0x7f9ec8159020 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.548+0000 7f9ec67fc700 1 -- 172.21.15.98:0/3667500320 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00092e0 con 0x7f9ec8159020 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec8159020 msgr2=0x7f9ec81594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec8159020 0x7f9ec81594a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009f20 tx=0x7f9ec0009840 comp rx=0 tx=0).stop 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3667500320 shutdown_connections 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec8159020 0x7f9ec81594a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3667500320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec8067b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3667500320 >> 172.21.15.98:0/3667500320 conn(0x7f9ec80cdc00 msgr2=0x7f9ec80c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:32.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3667500320 shutdown_connections 2023-10-06T01:40:32.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.549+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3667500320 wait complete. 2023-10-06T01:40:32.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.550+0000 7f9ecdbd5700 1 Processor -- start 2023-10-06T01:40:32.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.550+0000 7f9ecdbd5700 1 -- start start 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 0x7f9ec815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80afa30 con 0x7f9ec8159020 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec816adf0 con 0x7f9ec815a8a0 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ecdbd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80b1360 con 0x7f9ec80677b0 2023-10-06T01:40:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40838/0 (socket says 172.21.15.98:40838) 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 -- 172.21.15.98:0/3601698373 learned_addr learned my addr 172.21.15.98:0/3601698373 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec77fe700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec7fff700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 0x7f9ec815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 -- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 msgr2=0x7f9ec815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 -- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 msgr2=0x7f9ec815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 0x7f9ec815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 -- 172.21.15.98:0/3601698373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8009420 con 0x7f9ec8159020 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec7fff700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 0x7f9ec815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9ec6ffd700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c5b0 tx=0x7f9ebc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.551+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011590 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.552+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc011710 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.552+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec0009ae0 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.552+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011ad0 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.552+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec815c950 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.552+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec815c3e0 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.553+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ebc01b3b0 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.553+0000 7f9eaf7fe700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 0x7f9eb4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.553+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6515+0+0 (secure 0 0 0) 0x7f9ebc083dd0 con 0x7f9ec8159020 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.553+0000 7f9ec77fe700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 0x7f9eb4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.554+0000 7f9ec77fe700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 0x7f9eb4069460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80093f0 tx=0x7f9eb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.557+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ebc088070 con 0x7f9ec8159020 2023-10-06T01:40:32.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:32.740+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ec815a0a0 con 0x7f9ec8159020 2023-10-06T01:40:33.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.402+0000 7f9eaf7fe700 1 -- 172.21.15.98:0/3601698373 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9ebc017070 con 0x7f9ec8159020 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 msgr2=0x7f9eb4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 0x7f9eb4069460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80093f0 tx=0x7f9eb8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 msgr2=0x7f9ec815e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c5b0 tx=0x7f9ebc007f60 comp rx=0 tx=0).stop 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 shutdown_connections 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9eb4066fb0 0x7f9eb4069460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec815a8a0 0x7f9ec815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec80677b0 0x7f9ec815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 --2- 172.21.15.98:0/3601698373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec8159020 0x7f9ec815e380 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 >> 172.21.15.98:0/3601698373 conn(0x7f9ec80cdc00 msgr2=0x7f9ec80c1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:33.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 shutdown_connections 2023-10-06T01:40:33.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.406+0000 7f9ecdbd5700 1 -- 172.21.15.98:0/3601698373 wait complete. 2023-10-06T01:40:33.408 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2023-10-06T01:40:33.422 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-06T01:40:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.539+0000 7f6bf205d700 1 Processor -- start 2023-10-06T01:40:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.539+0000 7f6bf205d700 1 -- start start 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bf205d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bec0d0520 0x7f6bec0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bf205d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0d08f0 0x7f6bec0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bf205d700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec0b63b0 con 0x7f6bec142a50 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bf205d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec0b35f0 con 0x7f6bec0d0520 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bf205d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b5500 con 0x7f6bec0d08f0 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bec0d0520 0x7f6bec0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bebfff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6bec142a50 0x7f6bec142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41890/0 (socket says 172.21.15.98:41890) 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bebfff700 1 -- 172.21.15.98:0/2484462313 learned_addr learned my addr 172.21.15.98:0/2484462313 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beaffd700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0d08f0 0x7f6bec0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 -- 172.21.15.98:0/2484462313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0d08f0 msgr2=0x7f6bec0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0d08f0 0x7f6bec0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 -- 172.21.15.98:0/2484462313 >> v1:172.21.15.98:6789/0 conn(0x7f6bec142a50 legacy=0x7f6bec142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 -- 172.21.15.98:0/2484462313 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0b57a0 con 0x7f6bec0d0520 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6bea7fc700 1 -- 172.21.15.98:0/2484462313 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664308354 0 0) 0x7f6bec0b63b0 con 0x7f6bec142a50 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.540+0000 7f6beb7fe700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d0520 0x7f6bec0b25d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009910 tx=0x7f6bdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82e945a9365025e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bea7fc700 1 -- 172.21.15.98:0/2484462313 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00b820 con 0x7f6bec0d0520 2023-10-06T01:40:33.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bea7fc700 1 -- 172.21.15.98:0/2484462313 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00b980 con 0x7f6bec0d0520 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bea7fc700 1 -- 172.21.15.98:0/2484462313 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00bd20 con 0x7f6bec0d0520 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 -- 172.21.15.98:0/2484462313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d0520 msgr2=0x7f6bec0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d0520 0x7f6bec0b25d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009910 tx=0x7f6bdc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 -- 172.21.15.98:0/2484462313 shutdown_connections 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d0520 0x7f6bec0b25d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 --2- 172.21.15.98:0/2484462313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec0d08f0 0x7f6bec0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 -- 172.21.15.98:0/2484462313 >> 172.21.15.98:0/2484462313 conn(0x7f6bec06ed20 msgr2=0x7f6bec06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 -- 172.21.15.98:0/2484462313 shutdown_connections 2023-10-06T01:40:33.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.541+0000 7f6bf205d700 1 -- 172.21.15.98:0/2484462313 wait complete. 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.542+0000 7f6bf205d700 1 Processor -- start 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 -- start start 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0d0520 0x7f6bec0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 0x7f6bec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec16a880 con 0x7f6bec0d08f0 2023-10-06T01:40:33.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec16a9e0 con 0x7f6bec0d0520 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6bf205d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0cfa20 con 0x7f6bec142a50 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40846/0 (socket says 172.21.15.98:40846) 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.543+0000 7f6beaffd700 1 -- 172.21.15.98:0/4165536883 learned_addr learned my addr 172.21.15.98:0/4165536883 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beb7fe700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0d0520 0x7f6bec0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bebfff700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 0x7f6bec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 -- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 msgr2=0x7f6bec15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 0x7f6bec15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 -- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0d0520 msgr2=0x7f6bec0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0d0520 0x7f6bec0cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:33.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 -- 172.21.15.98:0/4165536883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc009420 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bebfff700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 0x7f6bec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6beaffd700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6be000d890 tx=0x7f6be0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000b600 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be000b760 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec0c53b0 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000bae0 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.544+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec0c5650 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.545+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec064ad0 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.545+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6be001e460 con 0x7f6bec0d08f0 2023-10-06T01:40:33.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.546+0000 7f6bd37fe700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 0x7f6bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.546+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6043+0+0 (secure 0 0 0) 0x7f6be0084ca0 con 0x7f6bec0d08f0 2023-10-06T01:40:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.546+0000 7f6beb7fe700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 0x7f6bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.546+0000 7f6beb7fe700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 0x7f6bd80693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0b1ee0 tx=0x7f6bdc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.550+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6be0051dc0 con 0x7f6bec0d08f0 2023-10-06T01:40:33.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:33.740+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6bec0b5a80 con 0x7f6bec0d08f0 2023-10-06T01:40:34.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.413+0000 7f6bd37fe700 1 -- 172.21.15.98:0/4165536883 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v28) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6be00553e0 con 0x7f6bec0d08f0 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 msgr2=0x7f6bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 0x7f6bd80693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0b1ee0 tx=0x7f6bdc017000 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 msgr2=0x7f6bec16a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6be000d890 tx=0x7f6be0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 shutdown_connections 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6bd8066f30 0x7f6bd80693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bec142a50 0x7f6bec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bec0d0520 0x7f6bec0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 --2- 172.21.15.98:0/4165536883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bec0d08f0 0x7f6bec16a340 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 >> 172.21.15.98:0/4165536883 conn(0x7f6bec06ed20 msgr2=0x7f6bec1417d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:34.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 shutdown_connections 2023-10-06T01:40:34.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.416+0000 7f6bf205d700 1 -- 172.21.15.98:0/4165536883 wait complete. 2023-10-06T01:40:34.418 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data_ec' removed 2023-10-06T01:40:34.431 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T01:40:34.431 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T01:40:34.432 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T01:40:34.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:34.432 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T01:40:34.432 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T01:40:34.432 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T01:40:34.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:34.433 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T01:40:34.433 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T01:40:34.433 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T01:40:34.433 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:34.433 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T01:40:34.433 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T01:40:34.433 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T01:40:34.434 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:34.441 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T01:40:34.442 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T01:40:34.442 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T01:40:34.442 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T01:40:34.443 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T01:40:34.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T01:40:34.444 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T01:40:34.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T01:40:34.484 INFO:tasks.ceph.mds.a:Started 2023-10-06T01:40:34.484 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:34.484 INFO:tasks.ceph.mds.c:Started 2023-10-06T01:40:34.485 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:34.486 INFO:tasks.ceph.mds.b:Started 2023-10-06T01:40:34.487 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:34.487 INFO:tasks.ceph.mds.d:Started 2023-10-06T01:40:34.487 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:34.488 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T01:40:34.559 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2023-10-06T01:40:34.560 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2023-10-06T01:40:34.567 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-06T01:40:34.567 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-06T01:40:34.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.610+0000 7f1345c3a700 1 Processor -- start 2023-10-06T01:40:34.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.610+0000 7f1345c3a700 1 -- start start 2023-10-06T01:40:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f1345c3a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13401595a0 0x7f13400aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f1345c3a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13400b49e0 0x7f13400aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f1345c3a700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13400b0ca0 con 0x7f13400b07a0 2023-10-06T01:40:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f1345c3a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13400b0f80 con 0x7f13400b49e0 2023-10-06T01:40:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f1345c3a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13400b1260 con 0x7f13401595a0 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13400b49e0 0x7f13400aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13400b49e0 0x7f13400aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40890/0 (socket says 172.21.15.98:40890) 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f13400b07a0 0x7f13400b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41910/0 (socket says 172.21.15.98:41910) 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 -- 172.21.15.98:0/3208526203 learned_addr learned my addr 172.21.15.98:0/3208526203 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133f7fe700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13401595a0 0x7f13400aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 -- 172.21.15.98:0/3208526203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13401595a0 msgr2=0x7f13400aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13401595a0 0x7f13400aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 -- 172.21.15.98:0/3208526203 >> v1:172.21.15.98:6789/0 conn(0x7f13400b07a0 legacy=0x7f13400b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133effd700 1 -- 172.21.15.98:0/3208526203 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400b1540 con 0x7f13400b49e0 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.611+0000 7f133e7fc700 1 -- 172.21.15.98:0/3208526203 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326195748 0 0) 0x7f13400b0ca0 con 0x7f13400b07a0 2023-10-06T01:40:34.614 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f133effd700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13400b49e0 0x7f13400aaab0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13340099b0 tx=0x7f1334009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cbb2695d3058b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f133e7fc700 1 -- 172.21.15.98:0/3208526203 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334009ea0 con 0x7f13400b49e0 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f133e7fc700 1 -- 172.21.15.98:0/3208526203 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133400b860 con 0x7f13400b49e0 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f133e7fc700 1 -- 172.21.15.98:0/3208526203 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133400bc00 con 0x7f13400b49e0 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 -- 172.21.15.98:0/3208526203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13400b49e0 msgr2=0x7f13400aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13400b49e0 0x7f13400aaab0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13340099b0 tx=0x7f1334009250 comp rx=0 tx=0).stop 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 -- 172.21.15.98:0/3208526203 shutdown_connections 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13401595a0 0x7f13400aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 --2- 172.21.15.98:0/3208526203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13400b49e0 0x7f13400aaab0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 -- 172.21.15.98:0/3208526203 >> 172.21.15.98:0/3208526203 conn(0x7f1340099830 msgr2=0x7f134006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 -- 172.21.15.98:0/3208526203 shutdown_connections 2023-10-06T01:40:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.612+0000 7f1345c3a700 1 -- 172.21.15.98:0/3208526203 wait complete. 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 Processor -- start 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 -- start start 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13400b07a0 0x7f134009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13400b49e0 0x7f134009afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134014c490 con 0x7f13401595a0 2023-10-06T01:40:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134009b950 con 0x7f13400b49e0 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f1345c3a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134009c4d0 con 0x7f13400b07a0 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f133ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f133ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40898/0 (socket says 172.21.15.98:40898) 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.614+0000 7f133ffff700 1 -- 172.21.15.98:0/2934751805 learned_addr learned my addr 172.21.15.98:0/2934751805 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133effd700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13400b49e0 0x7f134009afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133f7fe700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13400b07a0 0x7f134009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 -- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13400b07a0 msgr2=0x7f134009dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13400b07a0 0x7f134009dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 -- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13400b49e0 msgr2=0x7f134009afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13400b49e0 0x7f134009afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 -- 172.21.15.98:0/2934751805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340094f0 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f133ffff700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f133800c610 tx=0x7f1338007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338005a70 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338005bd0 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134009b500 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380070d0 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.615+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134009be80 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.616+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1340064ad0 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.616+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1338007440 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.617+0000 7f13277fe700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 0x7f132c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.617+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f1338083910 con 0x7f13401595a0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.617+0000 7f133f7fe700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 0x7f132c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.617+0000 7f133f7fe700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 0x7f132c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f133000f510 tx=0x7f133000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.621+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1338050be0 con 0x7f13401595a0 2023-10-06T01:40:34.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T01:40:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:34.819+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f134009c610 con 0x7f13401595a0 2023-10-06T01:40:35.428 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.427+0000 7f13277fe700 1 -- 172.21.15.98:0/2934751805 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1338017070 con 0x7f13401595a0 2023-10-06T01:40:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 msgr2=0x7f132c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 0x7f132c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f133000f510 tx=0x7f133000b040 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 msgr2=0x7f134015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f133800c610 tx=0x7f1338007d60 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 shutdown_connections 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f132c066f30 0x7f132c0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13401595a0 0x7f134015d1e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13400b07a0 0x7f134009dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 --2- 172.21.15.98:0/2934751805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13400b49e0 0x7f134009afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 >> 172.21.15.98:0/2934751805 conn(0x7f1340099830 msgr2=0x7f13400d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 shutdown_connections 2023-10-06T01:40:35.432 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.430+0000 7f1345c3a700 1 -- 172.21.15.98:0/2934751805 wait complete. 2023-10-06T01:40:35.432 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2023-10-06T01:40:35.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T01:40:35.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.558+0000 7f11d6297700 1 Processor -- start 2023-10-06T01:40:35.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.558+0000 7f11d6297700 1 -- start start 2023-10-06T01:40:35.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01595a0 0x7f11d00aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11d00b49e0 0x7f11d00aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f11d00b07a0 0x7f11d00b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00b0ca0 con 0x7f11d00b49e0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00b0f80 con 0x7f11d00b07a0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d6297700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00b1260 con 0x7f11d01595a0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11cf7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11d00b49e0 0x7f11d00aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11cf7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11d00b49e0 0x7f11d00aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40912/0 (socket says 172.21.15.98:40912) 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11cf7fe700 1 -- 172.21.15.98:0/866392911 learned_addr learned my addr 172.21.15.98:0/866392911 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11d4834700 1 --2- 172.21.15.98:0/866392911 >> v2:172.21.15.184:3300/0 conn(0x7f11d00b07a0 0x7f11d00b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.559+0000 7f11cffff700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01595a0 0x7f11d00aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 -- 172.21.15.98:0/866392911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01595a0 msgr2=0x7f11d00aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01595a0 0x7f11d00aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 -- 172.21.15.98:0/866392911 >> v2:172.21.15.184:3300/0 conn(0x7f11d00b07a0 msgr2=0x7f11d00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 --2- 172.21.15.98:0/866392911 >> v2:172.21.15.184:3300/0 conn(0x7f11d00b07a0 0x7f11d00b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 -- 172.21.15.98:0/866392911 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00b1540 con 0x7f11d00b49e0 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11cf7fe700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d00b49e0 0x7f11d00aaab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b2f50 tx=0x7f11c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996c73b0588f2b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11ceffd700 1 -- 172.21.15.98:0/866392911 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400b640 con 0x7f11d00b49e0 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11ceffd700 1 -- 172.21.15.98:0/866392911 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400b7a0 con 0x7f11d00b49e0 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11ceffd700 1 -- 172.21.15.98:0/866392911 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400bb60 con 0x7f11d00b49e0 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 -- 172.21.15.98:0/866392911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d00b49e0 msgr2=0x7f11d00aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d00b49e0 0x7f11d00aaab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b2f50 tx=0x7f11c4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 -- 172.21.15.98:0/866392911 shutdown_connections 2023-10-06T01:40:35.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 --2- 172.21.15.98:0/866392911 >> v2:172.21.15.184:3300/0 conn(0x7f11d00b07a0 0x7f11d00b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01595a0 0x7f11d00aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 --2- 172.21.15.98:0/866392911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d00b49e0 0x7f11d00aaab0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 -- 172.21.15.98:0/866392911 >> 172.21.15.98:0/866392911 conn(0x7f11d0099830 msgr2=0x7f11d006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:35.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.560+0000 7f11d6297700 1 -- 172.21.15.98:0/866392911 shutdown_connections 2023-10-06T01:40:35.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.561+0000 7f11d6297700 1 -- 172.21.15.98:0/866392911 wait complete. 2023-10-06T01:40:35.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.561+0000 7f11d6297700 1 Processor -- start 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 -- start start 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00b07a0 0x7f11d009d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 0x7f11d009af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d009d790 con 0x7f11d01595a0 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d009b480 con 0x7f11d00b07a0 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.562+0000 7f11d6297700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d009c2a0 con 0x7f11d00b49e0 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40914/0 (socket says 172.21.15.98:40914) 2023-10-06T01:40:35.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 -- 172.21.15.98:0/279206849 learned_addr learned my addr 172.21.15.98:0/279206849 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11cf7fe700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 0x7f11d009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 -- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 msgr2=0x7f11d009af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 0x7f11d009af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11cffff700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00b07a0 0x7f11d009d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 -- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00b07a0 msgr2=0x7f11d009d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00b07a0 0x7f11d009d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 -- 172.21.15.98:0/279206849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40094f0 con 0x7f11d01595a0 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11cf7fe700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 0x7f11d009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d4834700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009e90 tx=0x7f11c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c800cc20 con 0x7f11d01595a0 2023-10-06T01:40:35.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c800f040 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d009b690 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8014480 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.563+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d009bc50 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.564+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d0064ad0 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.565+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11c80147f0 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.565+0000 7f11b7fff700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 0x7f11bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.565+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f11c8085150 con 0x7f11d01595a0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.565+0000 7f11cffff700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 0x7f11bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.565+0000 7f11cffff700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 0x7f11bc0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009390 tx=0x7f11c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.569+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11c8055f00 con 0x7f11d01595a0 2023-10-06T01:40:35.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.829+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f11d00b1820 con 0x7f11d01595a0 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.829+0000 7f11b7fff700 1 -- 172.21.15.98:0/279206849 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f11c8055a90 con 0x7f11d01595a0 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.830 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9lPgAnLBAAjPpez3a4NMXMOAj1v9I0ug==", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2023-10-06T01:40:35.831 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9lKhKpNxAAynwQ7/OfekKfI1DgEHcXAA==", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9loeoKMxAAGFFjEHx40kNFdSV46mowsQ==", 2023-10-06T01:40:35.832 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAtZR9lcDnUAhAA/LRnWVjCf3o19pvkMLfjXQ==", 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.833 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAwZR9lbBlGHBAAq23Jb6irVfH1CrM8J54NGQ==", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.834 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAzZR9liLBvHBAABkFDdQyx6FyBrj7O4jKsQw==", 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA2ZR9lCFH5HhAAPe1eTYOZnHOb1K264zFY0A==", 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2023-10-06T01:40:35.836 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA5ZR9lyco2HxAAnSRZPU4vlmlkn4/obvlWeA==", 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA/ZR9lRZC1IxAAlrj4NHXrXoocVCesey7/TA==", 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.837 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBCZR9lnO3kIBAAvyVLJH3R3hrfMUQwBr2xpA==", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBFZR9lubpmIBAADMQvrtBlCG7dq7NCJDynqw==", 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.839 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBIZR9lBlpJHRAAX1mz3qw8W8JKNufZkolpkg==", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAtZR9lSDqnBhAAtiG2UleyTmLx7DKNixS1iA==", 2023-10-06T01:40:35.840 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAtZR9lOaUGDxAAjyXgHmQqX6NPEqANIZaUkg==", 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.841 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAtZR9lieoMDBAAi/Gm21ZFDpqR66eX6Hq1Ow==", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.842 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAtZR9lIXluFBAAGvd+AqaxvZ1SvnUacw8j4w==", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2023-10-06T01:40:35.843 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9lqDdoEBAAUqiUUIOmlmA42Uo7dcynVQ==", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.844 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mds", 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lznvTDhAA6A1TZiscquivg0ULi0R/5A==", 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T01:40:35.845 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lC4fTDhAAeLnh2A2erbRxI2jRVYshgg==", 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-osd", 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lo4/TDhAAMNnnHv8bAXMoWBDZ11eGRg==", 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lC5fTDhAA3CJgeeQAnQbI2snkY902Zg==", 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lbZ7TDhAAcBtv06robiFUuh5x1h5tCA==", 2023-10-06T01:40:35.847 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBOZR9lBabTDhAA+DnVHAebFQj2hIeOO4TOow==", 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.848 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9lJRf+KBAAg9krytJ71YkrNr+tMUBirg==", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAsZR9ljVHOJBAAhQ5RLlxGti1y0Db8NuigUA==", 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2023-10-06T01:40:35.849 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2023-10-06T01:40:35.850 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 msgr2=0x7f11bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 0x7f11bc0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009390 tx=0x7f11c0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 msgr2=0x7f11d00aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009e90 tx=0x7f11c800b040 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 shutdown_connections 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11bc066f30 0x7f11bc0693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11d01595a0 0x7f11d00aef90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00b07a0 0x7f11d009d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 --2- 172.21.15.98:0/279206849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d00b49e0 0x7f11d009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.831+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 >> 172.21.15.98:0/279206849 conn(0x7f11d0099830 msgr2=0x7f11d00697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:35.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.832+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 shutdown_connections 2023-10-06T01:40:35.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.832+0000 7f11d6297700 1 -- 172.21.15.98:0/279206849 wait complete. 2023-10-06T01:40:35.853 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T01:40:35.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.967+0000 7f7bd5170700 1 Processor -- start 2023-10-06T01:40:35.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.967+0000 7f7bd5170700 1 -- start start 2023-10-06T01:40:35.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bd5170700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bd00b49e0 0x7f7bd00aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bd5170700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bd00b07a0 0x7f7bd00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bd5170700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd00b0ca0 con 0x7f7bd01595a0 2023-10-06T01:40:35.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bd5170700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00b0f80 con 0x7f7bd00b49e0 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bd5170700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00b1260 con 0x7f7bd00b07a0 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bd00b49e0 0x7f7bd00aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bd00b49e0 0x7f7bd00aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40930/0 (socket says 172.21.15.98:40930) 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 -- 172.21.15.98:0/317649658 learned_addr learned my addr 172.21.15.98:0/317649658 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bcf59e700 1 --2- 172.21.15.98:0/317649658 >> v2:172.21.15.184:3300/0 conn(0x7f7bd00b07a0 0x7f7bd00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 -- 172.21.15.98:0/317649658 >> v2:172.21.15.184:3300/0 conn(0x7f7bd00b07a0 msgr2=0x7f7bd00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:35.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 --2- 172.21.15.98:0/317649658 >> v2:172.21.15.184:3300/0 conn(0x7f7bd00b07a0 0x7f7bd00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.968+0000 7f7bc659c700 1 -- 172.21.15.98:0/317649658 >> v1:172.21.15.184:6789/0 conn(0x7f7bd01595a0 legacy=0x7f7bd00aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bc659c700 1 -- 172.21.15.98:0/317649658 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd00b1540 con 0x7f7bd00b49e0 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bc659c700 1 --2- 172.21.15.98:0/317649658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd00aac80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80099b0 tx=0x7f7bb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5344814c227dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bce59c700 1 -- 172.21.15.98:0/317649658 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800b820 con 0x7f7bd00b49e0 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bce59c700 1 -- 172.21.15.98:0/317649658 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800b980 con 0x7f7bd00b49e0 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bce59c700 1 -- 172.21.15.98:0/317649658 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800bd20 con 0x7f7bd00b49e0 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 -- 172.21.15.98:0/317649658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 msgr2=0x7f7bd00aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 --2- 172.21.15.98:0/317649658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd00aac80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80099b0 tx=0x7f7bb8009250 comp rx=0 tx=0).stop 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 -- 172.21.15.98:0/317649658 shutdown_connections 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 --2- 172.21.15.98:0/317649658 >> v2:172.21.15.184:3300/0 conn(0x7f7bd00b07a0 0x7f7bd00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 --2- 172.21.15.98:0/317649658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd00aac80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 -- 172.21.15.98:0/317649658 >> 172.21.15.98:0/317649658 conn(0x7f7bd0099830 msgr2=0x7f7bd006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:35.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 -- 172.21.15.98:0/317649658 shutdown_connections 2023-10-06T01:40:35.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.969+0000 7f7bd5170700 1 -- 172.21.15.98:0/317649658 wait complete. 2023-10-06T01:40:35.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 Processor -- start 2023-10-06T01:40:35.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 -- start start 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bd00b07a0 0x7f7bd0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 0x7f7bd00abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015fd70 con 0x7f7bd00b49e0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd016a5c0 con 0x7f7bd01595a0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bd5170700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd00ac390 con 0x7f7bd00b07a0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.971+0000 7f7bc659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40938/0 (socket says 172.21.15.98:40938) 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 -- 172.21.15.98:0/2910619732 learned_addr learned my addr 172.21.15.98:0/2910619732 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bcf59e700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 0x7f7bd00abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bced9d700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bd00b07a0 0x7f7bd0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 -- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bd00b07a0 msgr2=0x7f7bd0158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bd00b07a0 0x7f7bd0158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 -- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 msgr2=0x7f7bd00abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 0x7f7bd00abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 -- 172.21.15.98:0/2910619732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80094f0 con 0x7f7bd00b49e0 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bcf59e700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 0x7f7bd00abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc659c700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80094c0 tx=0x7f7bb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8024440 con 0x7f7bd00b49e0 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb80245a0 con 0x7f7bd00b49e0 2023-10-06T01:40:35.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd015fa90 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8024960 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.972+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd00ab650 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.973+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd0064ad0 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.973+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bb801a420 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.974+0000 7f7bc77fe700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.974+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7bb8086330 con 0x7f7bd00b49e0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.974+0000 7f7bced9d700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:35.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.974+0000 7f7bced9d700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0001670 tx=0x7f7bc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:35.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:35.978+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bb80536d0 con 0x7f7bd00b49e0 2023-10-06T01:40:36.236 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.236+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7bd00b1820 con 0x7f7bd00b49e0 2023-10-06T01:40:36.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.239+0000 7f7bc77fe700 1 -- 172.21.15.98:0/2910619732 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7bb8056cf0 con 0x7f7bd00b49e0 2023-10-06T01:40:36.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 msgr2=0x7f7bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0001670 tx=0x7f7bc0009040 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 msgr2=0x7f7bd016a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80094c0 tx=0x7f7bb8017000 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 shutdown_connections 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bd01595a0 0x7f7bd00abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bd00b07a0 0x7f7bd0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 --2- 172.21.15.98:0/2910619732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bd00b49e0 0x7f7bd016a080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 >> 172.21.15.98:0/2910619732 conn(0x7f7bd0099830 msgr2=0x7f7bd0069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:36.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 shutdown_connections 2023-10-06T01:40:36.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.242+0000 7f7bd5170700 1 -- 172.21.15.98:0/2910619732 wait complete. 2023-10-06T01:40:36.254 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T01:40:36.369 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.368+0000 7f89e7e16700 1 Processor -- start 2023-10-06T01:40:36.369 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.368+0000 7f89e7e16700 1 -- start start 2023-10-06T01:40:36.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.369+0000 7f89e7e16700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e00691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.369+0000 7f89e7e16700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f89e015a220 0x7f89e00ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.369+0000 7f89e7e16700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e0068d50 con 0x7f89e0069720 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.369+0000 7f89e7e16700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e0154060 con 0x7f89e015a220 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.369+0000 7f89e7e16700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e015a6b0 con 0x7f89e00cb630 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f89e015a220 0x7f89e00ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f89e015a220 0x7f89e00ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40944/0 (socket says 172.21.15.98:40944) 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 -- 172.21.15.98:0/3993780578 learned_addr learned my addr 172.21.15.98:0/3993780578 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e5bb2700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e00691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 -- 172.21.15.98:0/3993780578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 msgr2=0x7f89e00691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e00691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 -- 172.21.15.98:0/3993780578 >> v1:172.21.15.184:6789/0 conn(0x7f89e0069720 legacy=0x7f89e0159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 -- 172.21.15.98:0/3993780578 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e015a990 con 0x7f89e015a220 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e63b3700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e015a220 0x7f89e00ca6b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f20 tx=0x7f89d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665bd3a613dad139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e4bb0700 1 -- 172.21.15.98:0/3993780578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d800e070 con 0x7f89e015a220 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e4bb0700 1 -- 172.21.15.98:0/3993780578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d800ba80 con 0x7f89e015a220 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.370+0000 7f89e4bb0700 1 -- 172.21.15.98:0/3993780578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80092e0 con 0x7f89e015a220 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 -- 172.21.15.98:0/3993780578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e015a220 msgr2=0x7f89e00ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e015a220 0x7f89e00ca6b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f20 tx=0x7f89d8009840 comp rx=0 tx=0).stop 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 -- 172.21.15.98:0/3993780578 shutdown_connections 2023-10-06T01:40:36.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e015a220 0x7f89e00ca6b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 --2- 172.21.15.98:0/3993780578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e00691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 -- 172.21.15.98:0/3993780578 >> 172.21.15.98:0/3993780578 conn(0x7f89e0070810 msgr2=0x7f89e0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:36.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 -- 172.21.15.98:0/3993780578 shutdown_connections 2023-10-06T01:40:36.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.371+0000 7f89e7e16700 1 -- 172.21.15.98:0/3993780578 wait complete. 2023-10-06T01:40:36.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.372+0000 7f89e7e16700 1 Processor -- start 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.372+0000 7f89e7e16700 1 -- start start 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015a220 0x7f89e00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c5860 con 0x7f89e0069720 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c5b00 con 0x7f89e015a220 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e7e16700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c5de0 con 0x7f89e00cb630 2023-10-06T01:40:36.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40960/0 (socket says 172.21.15.98:40960) 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 -- 172.21.15.98:0/2371460804 learned_addr learned my addr 172.21.15.98:0/2371460804 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e63b3700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015a220 0x7f89e00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e53b1700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 -- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 msgr2=0x7f89e01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e01529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 -- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015a220 msgr2=0x7f89e00cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015a220 0x7f89e00cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.373+0000 7f89e5bb2700 1 -- 172.21.15.98:0/2371460804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009ae0 con 0x7f89e0069720 2023-10-06T01:40:36.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89e5bb2700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d890 tx=0x7f89dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0125f0 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc012750 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e00c60c0 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc012af0 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.374+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e00c6680 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.375+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e00ccc10 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.375+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89dc01e460 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.376+0000 7f89d5ffb700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 0x7f89c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.376+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f89dc0849d0 con 0x7f89e0069720 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.376+0000 7f89e53b1700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 0x7f89c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.380 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.376+0000 7f89e53b1700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 0x7f89c40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89e01576b0 tx=0x7f89d0007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.381 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.380+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89dc055780 con 0x7f89e0069720 2023-10-06T01:40:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.639+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f89e0064ad0 con 0x7f89e0069720 2023-10-06T01:40:36.643 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.642+0000 7f89d5ffb700 1 -- 172.21.15.98:0/2371460804 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f89dc055310 con 0x7f89e0069720 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 msgr2=0x7f89c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 0x7f89c40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89e01576b0 tx=0x7f89d0007ec0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 msgr2=0x7f89e00c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d890 tx=0x7f89dc008040 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 shutdown_connections 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89c4066f30 0x7f89c40693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015a220 0x7f89e00cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89e0069720 0x7f89e00c5320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 --2- 172.21.15.98:0/2371460804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00cb630 0x7f89e01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 >> 172.21.15.98:0/2371460804 conn(0x7f89e0070810 msgr2=0x7f89e01538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:36.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 shutdown_connections 2023-10-06T01:40:36.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.645+0000 7f89e7e16700 1 -- 172.21.15.98:0/2371460804 wait complete. 2023-10-06T01:40:36.657 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T01:40:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.772+0000 7f4b71592700 1 Processor -- start 2023-10-06T01:40:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.772+0000 7f4b71592700 1 -- start start 2023-10-06T01:40:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b71592700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b71592700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b71592700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6c157e20 con 0x7f4b6c0ca3b0 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b71592700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0cbbd0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b71592700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0ceb50 con 0x7f4b6c0ce710 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b627fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b627fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40972/0 (socket says 172.21.15.98:40972) 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b627fc700 1 -- 172.21.15.98:0/3330898906 learned_addr learned my addr 172.21.15.98:0/3330898906 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.773+0000 7f4b6b7fe700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b627fc700 1 -- 172.21.15.98:0/3330898906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 msgr2=0x7f4b6c15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b627fc700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b627fc700 1 -- 172.21.15.98:0/3330898906 >> v1:172.21.15.184:6789/0 conn(0x7f4b6c0ca3b0 legacy=0x7f4b6c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b627fc700 1 -- 172.21.15.98:0/3330898906 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c0cb610 con 0x7f4b6c0adba0 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b627fc700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b540099b0 tx=0x7f4b54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aae0177e112632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b6a7fc700 1 -- 172.21.15.98:0/3330898906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54009ea0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b6a7fc700 1 -- 172.21.15.98:0/3330898906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400b860 con 0x7f4b6c0adba0 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b6a7fc700 1 -- 172.21.15.98:0/3330898906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400bc00 con 0x7f4b6c0adba0 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b71592700 1 -- 172.21.15.98:0/3330898906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 msgr2=0x7f4b6c0ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.774+0000 7f4b71592700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b540099b0 tx=0x7f4b54009250 comp rx=0 tx=0).stop 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 -- 172.21.15.98:0/3330898906 shutdown_connections 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 --2- 172.21.15.98:0/3330898906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0ce1d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 -- 172.21.15.98:0/3330898906 >> 172.21.15.98:0/3330898906 conn(0x7f4b6c070810 msgr2=0x7f4b6c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:36.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 -- 172.21.15.98:0/3330898906 shutdown_connections 2023-10-06T01:40:36.778 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.775+0000 7f4b71592700 1 -- 172.21.15.98:0/3330898906 wait complete. 2023-10-06T01:40:36.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.776+0000 7f4b71592700 1 Processor -- start 2023-10-06T01:40:36.780 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.776+0000 7f4b71592700 1 -- start start 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.776+0000 7f4b71592700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.776+0000 7f4b71592700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 0x7f4b6c167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.776+0000 7f4b71592700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b71592700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0c2c50 con 0x7f4b6c0adba0 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b71592700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0c2f20 con 0x7f4b6c0ca3b0 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b71592700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0c3200 con 0x7f4b6c0ce710 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40974/0 (socket says 172.21.15.98:40974) 2023-10-06T01:40:36.781 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 -- 172.21.15.98:0/1574392983 learned_addr learned my addr 172.21.15.98:0/1574392983 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b627fc700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 0x7f4b6c167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6b7fe700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 -- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 msgr2=0x7f4b6c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 -- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 msgr2=0x7f4b6c167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 0x7f4b6c167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 -- 172.21.15.98:0/1574392983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540094f0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b627fc700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 0x7f4b6c167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:36.782 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6b7fe700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b6affd700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00c590 tx=0x7f4b5c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c011590 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0116f0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c0c34e0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c011ab0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.777+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c14fa20 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.778+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c064ad0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.779+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b5c01b3b0 con 0x7f4b6c0adba0 2023-10-06T01:40:36.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.779+0000 7f4b637fe700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 0x7f4b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:36.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.779+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f4b5c083a30 con 0x7f4b6c0adba0 2023-10-06T01:40:36.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.779+0000 7f4b627fc700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 0x7f4b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:36.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.779+0000 7f4b627fc700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 0x7f4b4c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b540099b0 tx=0x7f4b54017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:36.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:36.783+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b5c050d50 con 0x7f4b6c0adba0 2023-10-06T01:40:37.040 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.039+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4b6c0cb8f0 con 0x7f4b6c0adba0 2023-10-06T01:40:37.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.043+0000 7f4b637fe700 1 -- 172.21.15.98:0/1574392983 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4b5c017070 con 0x7f4b6c0adba0 2023-10-06T01:40:37.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 msgr2=0x7f4b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 0x7f4b4c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b540099b0 tx=0x7f4b54017000 comp rx=0 tx=0).stop 2023-10-06T01:40:37.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 msgr2=0x7f4b6c0c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00c590 tx=0x7f4b5c00c950 comp rx=0 tx=0).stop 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 shutdown_connections 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b4c066f30 0x7f4b4c0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b6c0ce710 0x7f4b6c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b6c0adba0 0x7f4b6c0c45d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 --2- 172.21.15.98:0/1574392983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b6c0ca3b0 0x7f4b6c167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 >> 172.21.15.98:0/1574392983 conn(0x7f4b6c070810 msgr2=0x7f4b6c0ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 shutdown_connections 2023-10-06T01:40:37.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.045+0000 7f4b71592700 1 -- 172.21.15.98:0/1574392983 wait complete. 2023-10-06T01:40:37.057 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T01:40:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.170+0000 7fa25bdd5700 1 Processor -- start 2023-10-06T01:40:37.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.170+0000 7fa25bdd5700 1 -- start start 2023-10-06T01:40:37.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25bdd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b49e0 0x7fa2540aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25bdd5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2540b07a0 0x7fa2540b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25bdd5700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2540b0ca0 con 0x7fa2541595a0 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25bdd5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2540b0f80 con 0x7fa2540b07a0 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25bdd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b1260 con 0x7fa2540b49e0 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa25a372700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2540b07a0 0x7fa2540b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa259b71700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa2541595a0 0x7fa2540aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41916/0 (socket says 172.21.15.98:41916) 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.171+0000 7fa259b71700 1 -- 172.21.15.98:0/3985342390 learned_addr learned my addr 172.21.15.98:0/3985342390 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25a372700 1 -- 172.21.15.98:0/3985342390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b49e0 msgr2=0x7fa2540aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa259370700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b49e0 0x7fa2540aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa258b6f700 1 -- 172.21.15.98:0/3985342390 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978363944 0 0) 0x7fa2540b0ca0 con 0x7fa2541595a0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25a372700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b49e0 0x7fa2540aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25a372700 1 -- 172.21.15.98:0/3985342390 >> v1:172.21.15.98:6789/0 conn(0x7fa2541595a0 legacy=0x7fa2540aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25a372700 1 -- 172.21.15.98:0/3985342390 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540b1540 con 0x7fa2540b07a0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25a372700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2540b07a0 0x7fa2540b1c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0098e0 tx=0x7fa24c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f08a8e3e6eee5fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa258b6f700 1 -- 172.21.15.98:0/3985342390 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c009e80 con 0x7fa2540b07a0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa258b6f700 1 -- 172.21.15.98:0/3985342390 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c00b860 con 0x7fa2540b07a0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa258b6f700 1 -- 172.21.15.98:0/3985342390 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00bbe0 con 0x7fa2540b07a0 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 -- 172.21.15.98:0/3985342390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2540b07a0 msgr2=0x7fa2540b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2540b07a0 0x7fa2540b1c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0098e0 tx=0x7fa24c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 -- 172.21.15.98:0/3985342390 shutdown_connections 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2540b07a0 0x7fa2540b1c00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/3985342390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b49e0 0x7fa2540aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.172+0000 7fa25bdd5700 1 -- 172.21.15.98:0/3985342390 >> 172.21.15.98:0/3985342390 conn(0x7fa254099830 msgr2=0x7fa25406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.173+0000 7fa25bdd5700 1 -- 172.21.15.98:0/3985342390 shutdown_connections 2023-10-06T01:40:37.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.173+0000 7fa25bdd5700 1 -- 172.21.15.98:0/3985342390 wait complete. 2023-10-06T01:40:37.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 Processor -- start 2023-10-06T01:40:37.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 -- start start 2023-10-06T01:40:37.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b07a0 0x7fa254158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2540b49e0 0x7fa25415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25415aa40 con 0x7fa2541595a0 2023-10-06T01:40:37.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa254159e30 con 0x7fa2540b49e0 2023-10-06T01:40:37.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25bdd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25415a110 con 0x7fa2540b07a0 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25a372700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25a372700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40988/0 (socket says 172.21.15.98:40988) 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.174+0000 7fa25a372700 1 -- 172.21.15.98:0/2833376914 learned_addr learned my addr 172.21.15.98:0/2833376914 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa259b71700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b07a0 0x7fa254158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa259370700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2540b49e0 0x7fa25415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 -- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b07a0 msgr2=0x7fa254158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b07a0 0x7fa254158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 -- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2540b49e0 msgr2=0x7fa25415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2540b49e0 0x7fa25415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 -- 172.21.15.98:0/2833376914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c009420 con 0x7fa2541595a0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25a372700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b3c0 tx=0x7fa24c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c011630 con 0x7fa2541595a0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c011790 con 0x7fa2541595a0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25415a3f0 con 0x7fa2541595a0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c011b50 con 0x7fa2541595a0 2023-10-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.175+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25415cf20 con 0x7fa2541595a0 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.176+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25415b410 con 0x7fa2541595a0 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.177+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa24c0253f0 con 0x7fa2541595a0 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.177+0000 7fa249ffb700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 0x7fa2380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.177+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fa24c086180 con 0x7fa2541595a0 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.177+0000 7fa259b71700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 0x7fa2380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.177+0000 7fa259b71700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 0x7fa2380693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa250002ef0 tx=0x7fa250009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.181+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa24c0534a0 con 0x7fa2541595a0 2023-10-06T01:40:37.438 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.438+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa254064ad0 con 0x7fa2541595a0 2023-10-06T01:40:37.441 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.441+0000 7fa249ffb700 1 -- 172.21.15.98:0/2833376914 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa24c056ac0 con 0x7fa2541595a0 2023-10-06T01:40:37.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 msgr2=0x7fa2380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 0x7fa2380693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa250002ef0 tx=0x7fa250009040 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 msgr2=0x7fa25409d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b3c0 tx=0x7fa24c017000 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 shutdown_connections 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa238066f30 0x7fa2380693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2541595a0 0x7fa25409d520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2540b07a0 0x7fa254158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 --2- 172.21.15.98:0/2833376914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2540b49e0 0x7fa25415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 >> 172.21.15.98:0/2833376914 conn(0x7fa254099830 msgr2=0x7fa2540b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 shutdown_connections 2023-10-06T01:40:37.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.443+0000 7fa25bdd5700 1 -- 172.21.15.98:0/2833376914 wait complete. 2023-10-06T01:40:37.455 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T01:40:37.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.569+0000 7f910f59e700 1 Processor -- start 2023-10-06T01:40:37.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.569+0000 7f910f59e700 1 -- start start 2023-10-06T01:40:37.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f9110069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910f59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f911015a430 0x7f911006cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910f59e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9110159fa0 con 0x7f911006d160 2023-10-06T01:40:37.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910f59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91101599c0 con 0x7f911015a430 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911006d650 con 0x7f91100696d0 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910ed9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f911006d160 0x7f91100b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41918/0 (socket says 172.21.15.98:41918) 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910ed9d700 1 -- 172.21.15.98:0/861389307 learned_addr learned my addr 172.21.15.98:0/861389307 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f9105d9b700 1 --2- 172.21.15.98:0/861389307 >> v2:172.21.15.184:3300/0 conn(0x7f911015a430 0x7f911006cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714472343 0 0) 0x7f9110159fa0 con 0x7f911006d160 2023-10-06T01:40:37.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910e59c700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f9110069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.570+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90fc003570 con 0x7f911006d160 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892686301 0 0) 0x7f90fc003570 con 0x7f911006d160 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 msgr2=0x7f9110069aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f9110069aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 >> v2:172.21.15.184:3300/0 conn(0x7f911015a430 msgr2=0x7f911006cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 --2- 172.21.15.98:0/861389307 >> v2:172.21.15.184:3300/0 conn(0x7f911015a430 0x7f911006cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911006d930 con 0x7f911006d160 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f91080017f0 con 0x7f911006d160 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 >> v1:172.21.15.98:6789/0 conn(0x7f911006d160 legacy=0x7f91100b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc003a60 0x7f90fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004810 0x7f90fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f90fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91101599c0 con 0x7f90fc003a60 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911006d650 con 0x7f90fc004810 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc00ba60 con 0x7f91100696d0 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc003a60 0x7f90fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910ed9d700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f90fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f9105d9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004810 0x7f90fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 -- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 msgr2=0x7f90fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f90fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 -- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004810 msgr2=0x7f90fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004810 0x7f90fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 -- 172.21.15.98:0/861389307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc003570 con 0x7f90fc003a60 2023-10-06T01:40:37.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910ed9d700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100696d0 0x7f90fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f9105d9b700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004810 0x7f90fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910e59c700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc003a60 0x7f90fc003e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9100009c30 tx=0x7f91000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f9eb06ab838ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000e710 con 0x7f90fc003a60 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910000e870 con 0x7f90fc003a60 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.571+0000 7f910dd9b700 1 -- 172.21.15.98:0/861389307 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000ec30 con 0x7f90fc003a60 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 -- 172.21.15.98:0/861389307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc003a60 msgr2=0x7f90fc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 --2- 172.21.15.98:0/861389307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc003a60 0x7f90fc003e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9100009c30 tx=0x7f91000091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910ed9d700 1 -- 172.21.15.98:0/861389307 reap_dead start 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 -- 172.21.15.98:0/861389307 shutdown_connections 2023-10-06T01:40:37.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 -- 172.21.15.98:0/861389307 >> 172.21.15.98:0/861389307 conn(0x7f911006b540 msgr2=0x7f911006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 -- 172.21.15.98:0/861389307 shutdown_connections 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.572+0000 7f910f59e700 1 -- 172.21.15.98:0/861389307 wait complete. 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.573+0000 7f910f59e700 1 Processor -- start 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.573+0000 7f910f59e700 1 -- start start 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003a60 0x7f911013dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 0x7f91100aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110152090 con 0x7f9110150080 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911013e100 con 0x7f90fc003a60 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9110150a30 con 0x7f91100ae8c0 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40994/0 (socket says 172.21.15.98:40994) 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 -- 172.21.15.98:0/633589655 learned_addr learned my addr 172.21.15.98:0/633589655 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910e59c700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003a60 0x7f911013dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910ed9d700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 0x7f91100aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 -- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 msgr2=0x7f91100aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 0x7f91100aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 -- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003a60 msgr2=0x7f911013dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003a60 0x7f911013dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 -- 172.21.15.98:0/633589655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911014e3b0 con 0x7f9110150080 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f910ed9d700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 0x7f91100aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.574+0000 7f9105d9b700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f90f8011940 tx=0x7f90f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800f730 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f800f890 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91100af5c0 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800fc30 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911014dd60 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.575+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110071440 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.576+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90f8015460 con 0x7f9110150080 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.576+0000 7f9106ffd700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 0x7f90f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.576+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f90f8084be0 con 0x7f9110150080 2023-10-06T01:40:37.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.576+0000 7f910e59c700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 0x7f90f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.577+0000 7f910e59c700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 0x7f90f00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f91000091e0 tx=0x7f910001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.580+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90f8051f00 con 0x7f9110150080 2023-10-06T01:40:37.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.840+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9110159c30 con 0x7f9110150080 2023-10-06T01:40:37.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.843+0000 7f9106ffd700 1 -- 172.21.15.98:0/633589655 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90f8055520 con 0x7f9110150080 2023-10-06T01:40:37.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.845+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 msgr2=0x7f90f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.845+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 0x7f90f00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f91000091e0 tx=0x7f910001a000 comp rx=0 tx=0).stop 2023-10-06T01:40:37.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.845+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 msgr2=0x7f91101504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.845+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f90f8011940 tx=0x7f90f8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 shutdown_connections 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f90f0066f30 0x7f90f00693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91100ae8c0 0x7f91100aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003a60 0x7f911013dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 --2- 172.21.15.98:0/633589655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9110150080 0x7f91101504f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 >> 172.21.15.98:0/633589655 conn(0x7f911006b540 msgr2=0x7f911014d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 shutdown_connections 2023-10-06T01:40:37.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.846+0000 7f910f59e700 1 -- 172.21.15.98:0/633589655 wait complete. 2023-10-06T01:40:37.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T01:40:37.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.973+0000 7fa479dd5700 1 Processor -- start 2023-10-06T01:40:37.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.973+0000 7fa479dd5700 1 -- start start 2023-10-06T01:40:37.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa479dd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 0x7fa4740b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa479dd5700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740cb7e0 con 0x7fa4740b3610 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa479dd5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740b08a0 con 0x7fa4740b2140 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa479dd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740cb190 con 0x7fa474159480 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa46bfff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa4740b3610 0x7fa4740cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41920/0 (socket says 172.21.15.98:41920) 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa46bfff700 1 -- 172.21.15.98:0/3374283034 learned_addr learned my addr 172.21.15.98:0/3374283034 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3374283034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 0x7fa4740b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.974+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254112309 0 0) 0x7fa4740cb7e0 con 0x7fa4740b3610 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa45c003570 con 0x7fa4740b3610 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178363688 0 0) 0x7fa4740b08a0 con 0x7fa4740b2140 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4740cb7e0 con 0x7fa4740b2140 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa478dd3700 1 -- 172.21.15.98:0/3374283034 >> v1:172.21.15.184:6789/0 conn(0x7fa4740b2140 legacy=0x7fa4740b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa478dd3700 1 -- 172.21.15.98:0/3374283034 >> v1:172.21.15.98:6789/0 conn(0x7fa4740b3610 legacy=0x7fa4740cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa478dd3700 1 -- 172.21.15.98:0/3374283034 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740cb470 con 0x7fa474159480 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3374283034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 0x7fa4740b30d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa470009910 tx=0x7fa4700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482a2edff22f7d5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470011630 con 0x7fa474159480 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa470011790 con 0x7fa474159480 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.975+0000 7fa46b7fe700 1 -- 172.21.15.98:0/3374283034 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470011b10 con 0x7fa474159480 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 -- 172.21.15.98:0/3374283034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 msgr2=0x7fa4740b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3374283034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 0x7fa4740b30d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa470009910 tx=0x7fa4700091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 -- 172.21.15.98:0/3374283034 shutdown_connections 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3374283034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa474159480 0x7fa4740b30d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 -- 172.21.15.98:0/3374283034 >> 172.21.15.98:0/3374283034 conn(0x7fa47406ed20 msgr2=0x7fa47406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 -- 172.21.15.98:0/3374283034 shutdown_connections 2023-10-06T01:40:37.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.976+0000 7fa479dd5700 1 -- 172.21.15.98:0/3374283034 wait complete. 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.977+0000 7fa479dd5700 1 Processor -- start 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.977+0000 7fa479dd5700 1 -- start start 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 0x7fa47415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 0x7fa4740c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47416a1d0 con 0x7fa4740b3610 2023-10-06T01:40:37.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47415c940 con 0x7fa474159480 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa479dd5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741695c0 con 0x7fa4740b2140 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40996/0 (socket says 172.21.15.98:40996) 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 -- 172.21.15.98:0/3230523541 learned_addr learned my addr 172.21.15.98:0/3230523541 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa4795d4700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 0x7fa4740c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 0x7fa47415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 -- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 msgr2=0x7fa47415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 0x7fa47415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 -- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 msgr2=0x7fa4740c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 0x7fa4740c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:37.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 -- 172.21.15.98:0/3230523541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470009420 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa4795d4700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 0x7fa4740c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 0x7fa47415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa46bfff700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c790 tx=0x7fa464007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464005b30 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.978+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464005c90 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.979+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464007300 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.979+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4741698a0 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.979+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa474169e60 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.979+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4740c7310 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.980+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa46401b400 con 0x7fa4740b3610 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.980+0000 7fa468ff9700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:37.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.980+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fa464084040 con 0x7fa4740b3610 2023-10-06T01:40:37.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.980+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:37.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.981+0000 7fa478dd3700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 0x7fa4600693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa470009860 tx=0x7fa470017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:37.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:37.984+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4740c7310 con 0x7fa4740b3610 2023-10-06T01:40:38.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.242+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa4740c7310 con 0x7fa4740b3610 2023-10-06T01:40:38.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.245+0000 7fa468ff9700 1 -- 172.21.15.98:0/3230523541 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa4740c7310 con 0x7fa4740b3610 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 msgr2=0x7fa4600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 0x7fa4600693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa470009860 tx=0x7fa470017000 comp rx=0 tx=0).stop 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 msgr2=0x7fa4740aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c790 tx=0x7fa464007ea0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 shutdown_connections 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa474159480 0x7fa4740c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b2140 0x7fa47415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 --2- 172.21.15.98:0/3230523541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4740b3610 0x7fa4740aefe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 >> 172.21.15.98:0/3230523541 conn(0x7fa47406ed20 msgr2=0x7fa4741551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:38.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 shutdown_connections 2023-10-06T01:40:38.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.247+0000 7fa479dd5700 1 -- 172.21.15.98:0/3230523541 wait complete. 2023-10-06T01:40:38.260 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:38.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.373+0000 7f4638a24700 1 Processor -- start 2023-10-06T01:40:38.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.373+0000 7f4638a24700 1 -- start start 2023-10-06T01:40:38.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.374+0000 7f4638a24700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f46341595a0 0x7f46340aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.374+0000 7f4638a24700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46340aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.374+0000 7f4638a24700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46340b0ca0 con 0x7f46340b07a0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.374+0000 7f4638a24700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340b0f80 con 0x7f46341595a0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.374+0000 7f4638a24700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b1260 con 0x7f46340b49e0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f46341595a0 0x7f46340aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f4632d9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f46340b07a0 0x7f46340b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41924/0 (socket says 172.21.15.98:41924) 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f4632d9d700 1 -- 172.21.15.98:0/2560367090 learned_addr learned my addr 172.21.15.98:0/2560367090 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f4631d9b700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46340aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 -- 172.21.15.98:0/2560367090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 msgr2=0x7f46340aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46340aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 -- 172.21.15.98:0/2560367090 >> v1:172.21.15.98:6789/0 conn(0x7f46340b07a0 legacy=0x7f46340b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 -- 172.21.15.98:0/2560367090 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340b1540 con 0x7f46341595a0 2023-10-06T01:40:38.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463159a700 1 -- 172.21.15.98:0/2560367090 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146518663 0 0) 0x7f46340b0ca0 con 0x7f46340b07a0 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463259c700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f46340aa570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4624009eb0 tx=0x7f4624009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d5089e9b015b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463159a700 1 -- 172.21.15.98:0/2560367090 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400e070 con 0x7f46341595a0 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463159a700 1 -- 172.21.15.98:0/2560367090 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400bc30 con 0x7f46341595a0 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.375+0000 7f463159a700 1 -- 172.21.15.98:0/2560367090 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624012430 con 0x7f46341595a0 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 -- 172.21.15.98:0/2560367090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 msgr2=0x7f46340aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f46340aa570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4624009eb0 tx=0x7f4624009780 comp rx=0 tx=0).stop 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 -- 172.21.15.98:0/2560367090 shutdown_connections 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f46340aa570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 --2- 172.21.15.98:0/2560367090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46340aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 -- 172.21.15.98:0/2560367090 >> 172.21.15.98:0/2560367090 conn(0x7f4634099830 msgr2=0x7f463406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:38.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 -- 172.21.15.98:0/2560367090 shutdown_connections 2023-10-06T01:40:38.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.376+0000 7f4638a24700 1 -- 172.21.15.98:0/2560367090 wait complete. 2023-10-06T01:40:38.381 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.377+0000 7f4638a24700 1 Processor -- start 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.377+0000 7f4638a24700 1 -- start start 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.377+0000 7f4638a24700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 0x7f463415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.377+0000 7f4638a24700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46341587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.377+0000 7f4638a24700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4638a24700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463409dfc0 con 0x7f46341595a0 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4638a24700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463416c960 con 0x7f46340b07a0 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4638a24700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463416cc40 con 0x7f46340b49e0 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41018/0 (socket says 172.21.15.98:41018) 2023-10-06T01:40:38.382 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 -- 172.21.15.98:0/1000295721 learned_addr learned my addr 172.21.15.98:0/1000295721 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f463259c700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 0x7f463415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4631d9b700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46341587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 -- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 msgr2=0x7f46341587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46341587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 -- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 msgr2=0x7f463415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 0x7f463415c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 -- 172.21.15.98:0/1000295721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009a20 con 0x7f46341595a0 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f463259c700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 0x7f463415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4632d9d700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c7e0 tx=0x7f462c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005c00 con 0x7f46341595a0 2023-10-06T01:40:38.383 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c005d60 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463416cf20 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c004a80 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.378+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463416d4e0 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.379+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463416d700 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.380+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f462c005030 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.380+0000 7f46227fc700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 0x7f46140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.380+0000 7f463259c700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 0x7f46140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.380+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f462c0838c0 con 0x7f46341595a0 2023-10-06T01:40:38.384 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.380+0000 7f463259c700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 0x7f46140693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4624009eb0 tx=0x7f4624017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.385 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.384+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f462c04b5a0 con 0x7f46341595a0 2023-10-06T01:40:38.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.571+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4634064ad0 con 0x7f46341595a0 2023-10-06T01:40:38.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.571+0000 7f46227fc700 1 -- 172.21.15.98:0/1000295721 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+13951 (secure 0 0 0) 0x7f462c050be0 con 0x7f46341595a0 2023-10-06T01:40:38.572 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:38.572 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":32,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:40:35.436500+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:38.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 msgr2=0x7f46140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 0x7f46140693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4624009eb0 tx=0x7f4624017000 comp rx=0 tx=0).stop 2023-10-06T01:40:38.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 msgr2=0x7f4634156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.574 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c7e0 tx=0x7f462c007d90 comp rx=0 tx=0).stop 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 shutdown_connections 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4614066f30 0x7f46140693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46341595a0 0x7f4634156320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46340b07a0 0x7f463415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 --2- 172.21.15.98:0/1000295721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46340b49e0 0x7f46341587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.573+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 >> 172.21.15.98:0/1000295721 conn(0x7f4634099830 msgr2=0x7f4634069670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.574+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 shutdown_connections 2023-10-06T01:40:38.575 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.574+0000 7f4638a24700 1 -- 172.21.15.98:0/1000295721 wait complete. 2023-10-06T01:40:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T01:40:38.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:40:38.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.703+0000 7fbce3388700 1 Processor -- start 2023-10-06T01:40:38.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.703+0000 7fbce3388700 1 -- start start 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce3388700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbcdc159510 0x7fbcdc0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce3388700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcdc1598e0 0x7fbcdc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce3388700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc0cf370 con 0x7fbcdc0ce820 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce3388700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc0cf610 con 0x7fbcdc159510 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce3388700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc0cf8f0 con 0x7fbcdc1598e0 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbcdc159510 0x7fbcdc0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1925700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbcdc0ce820 0x7fbcdc0cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41938/0 (socket says 172.21.15.98:41938) 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1925700 1 -- 172.21.15.98:0/840546500 learned_addr learned my addr 172.21.15.98:0/840546500 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:38.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 -- 172.21.15.98:0/840546500 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc1598e0 msgr2=0x7fbcdc0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 --2- 172.21.15.98:0/840546500 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc1598e0 0x7fbcdc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 -- 172.21.15.98:0/840546500 >> v1:172.21.15.98:6789/0 conn(0x7fbcdc0ce820 legacy=0x7fbcdc0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 -- 172.21.15.98:0/840546500 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0cfbd0 con 0x7fbcdc159510 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbcdbfff700 1 -- 172.21.15.98:0/840546500 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966250238 0 0) 0x7fbcdc0cf370 con 0x7fbcdc0ce820 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbce1124700 1 --2- 172.21.15.98:0/840546500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc0cb5a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0c73b0 tx=0x7fbcd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e2cdd798e831a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbcdbfff700 1 -- 172.21.15.98:0/840546500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400b6b0 con 0x7fbcdc159510 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbcdbfff700 1 -- 172.21.15.98:0/840546500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd400b830 con 0x7fbcdc159510 2023-10-06T01:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.704+0000 7fbcdbfff700 1 -- 172.21.15.98:0/840546500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400bbf0 con 0x7fbcdc159510 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 -- 172.21.15.98:0/840546500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 msgr2=0x7fbcdc0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 --2- 172.21.15.98:0/840546500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc0cb5a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0c73b0 tx=0x7fbcd40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 -- 172.21.15.98:0/840546500 shutdown_connections 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 --2- 172.21.15.98:0/840546500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc0cb5a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 --2- 172.21.15.98:0/840546500 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc1598e0 0x7fbcdc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 -- 172.21.15.98:0/840546500 >> 172.21.15.98:0/840546500 conn(0x7fbcdc089e90 msgr2=0x7fbcdc0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 -- 172.21.15.98:0/840546500 shutdown_connections 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.705+0000 7fbce3388700 1 -- 172.21.15.98:0/840546500 wait complete. 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.706+0000 7fbce3388700 1 Processor -- start 2023-10-06T01:40:38.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.706+0000 7fbce3388700 1 -- start start 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc0ce820 0x7fbcdc157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 0x7fbcdc068600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0b6380 con 0x7fbcdc159510 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc157560 con 0x7fbcdc0ce820 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce3388700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc16df50 con 0x7fbcdc1598e0 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41034/0 (socket says 172.21.15.98:41034) 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 -- 172.21.15.98:0/899313676 learned_addr learned my addr 172.21.15.98:0/899313676 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce1925700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 0x7fbcdc068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce1124700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc0ce820 0x7fbcdc157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 -- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 msgr2=0x7fbcdc068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 0x7fbcdc068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 -- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc0ce820 msgr2=0x7fbcdc157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc0ce820 0x7fbcdc157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 -- 172.21.15.98:0/899313676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4009420 con 0x7fbcdc159510 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce1925700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 0x7fbcdc068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.707+0000 7fbce0923700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000bb00 tx=0x7fbcd00097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.708+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000c900 con 0x7fbcdc159510 2023-10-06T01:40:38.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.708+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd000ca60 con 0x7fbcdc159510 2023-10-06T01:40:38.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.708+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc0b3640 con 0x7fbcdc159510 2023-10-06T01:40:38.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.708+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000ce20 con 0x7fbcdc159510 2023-10-06T01:40:38.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.708+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc0b5dc0 con 0x7fbcdc159510 2023-10-06T01:40:38.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.709+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc064ad0 con 0x7fbcdc159510 2023-10-06T01:40:38.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.709+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcd0014620 con 0x7fbcdc159510 2023-10-06T01:40:38.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.709+0000 7fbcd97fa700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:38.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.709+0000 7fbce1124700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:38.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.709+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fbcd0084440 con 0x7fbcdc159510 2023-10-06T01:40:38.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.710+0000 7fbce1124700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 0x7fbcc80693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0c73b0 tx=0x7fbcd4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:38.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.714+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcd004c5a0 con 0x7fbcdc159510 2023-10-06T01:40:38.894 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.894+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbcdc0cfeb0 con 0x7fbcdc159510 2023-10-06T01:40:38.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.894+0000 7fbcd97fa700 1 -- 172.21.15.98:0/899313676 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbcd0051760 con 0x7fbcdc159510 2023-10-06T01:40:38.895 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 msgr2=0x7fbcc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 0x7fbcc80693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0c73b0 tx=0x7fbcd4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 msgr2=0x7fbcdc16da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000bb00 tx=0x7fbcd00097b0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 shutdown_connections 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.897 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcdc1598e0 0x7fbcdc068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc0ce820 0x7fbcdc157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 --2- 172.21.15.98:0/899313676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc159510 0x7fbcdc16da10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:38.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 >> 172.21.15.98:0/899313676 conn(0x7fbcdc089e90 msgr2=0x7fbcdc15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:38.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 shutdown_connections 2023-10-06T01:40:38.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:38.896+0000 7fbce3388700 1 -- 172.21.15.98:0/899313676 wait complete. 2023-10-06T01:40:38.908 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:40:39.025 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.025+0000 7ff484e4a700 1 Processor -- start 2023-10-06T01:40:39.026 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.025+0000 7ff484e4a700 1 -- start start 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff484e4a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 0x7ff4800cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff484e4a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff4800cbae0 0x7ff4800cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff484e4a700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4800cbf00 con 0x7ff4801598e0 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff484e4a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4800cf610 con 0x7ff4800cbae0 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff484e4a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800cf8f0 con 0x7ff480159510 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff4800cbae0 0x7ff4800cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47dd9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff4801598e0 0x7ff4800ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41948/0 (socket says 172.21.15.98:41948) 2023-10-06T01:40:39.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47dd9b700 1 -- 172.21.15.98:0/2196706295 learned_addr learned my addr 172.21.15.98:0/2196706295 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47e59c700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 0x7ff4800cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 -- 172.21.15.98:0/2196706295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 msgr2=0x7ff4800cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 0x7ff4800cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 -- 172.21.15.98:0/2196706295 >> v1:172.21.15.98:6789/0 conn(0x7ff4801598e0 legacy=0x7ff4800ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47d59a700 1 -- 172.21.15.98:0/2196706295 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528101642 0 0) 0x7ff4800cbf00 con 0x7ff4801598e0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 -- 172.21.15.98:0/2196706295 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800cfbd0 con 0x7ff4800cbae0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47e59c700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 0x7ff4800cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47ed9d700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800cbae0 0x7ff4800cef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e60 tx=0x7ff470009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8944197c06bc2a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47d59a700 1 -- 172.21.15.98:0/2196706295 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000e070 con 0x7ff4800cbae0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47d59a700 1 -- 172.21.15.98:0/2196706295 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47000bc30 con 0x7ff4800cbae0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.026+0000 7ff47d59a700 1 -- 172.21.15.98:0/2196706295 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470012430 con 0x7ff4800cbae0 2023-10-06T01:40:39.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 -- 172.21.15.98:0/2196706295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800cbae0 msgr2=0x7ff4800cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800cbae0 0x7ff4800cef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e60 tx=0x7ff470009780 comp rx=0 tx=0).stop 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 -- 172.21.15.98:0/2196706295 shutdown_connections 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800cbae0 0x7ff4800cef30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 --2- 172.21.15.98:0/2196706295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff480159510 0x7ff4800cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 -- 172.21.15.98:0/2196706295 >> 172.21.15.98:0/2196706295 conn(0x7ff480089e90 msgr2=0x7ff4800997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 -- 172.21.15.98:0/2196706295 shutdown_connections 2023-10-06T01:40:39.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.027+0000 7ff484e4a700 1 -- 172.21.15.98:0/2196706295 wait complete. 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.028+0000 7ff484e4a700 1 Processor -- start 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.028+0000 7ff484e4a700 1 -- start start 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4800cbae0 0x7ff4800c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4801598e0 0x7ff480069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015b4a0 con 0x7ff480159510 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015ba20 con 0x7ff4801598e0 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff484e4a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015f320 con 0x7ff4800cbae0 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41064/0 (socket says 172.21.15.98:41064) 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 -- 172.21.15.98:0/574883050 learned_addr learned my addr 172.21.15.98:0/574883050 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47e59c700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4800cbae0 0x7ff4800c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47ed9d700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4801598e0 0x7ff480069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 -- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4800cbae0 msgr2=0x7ff4800c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4800cbae0 0x7ff4800c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 -- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4801598e0 msgr2=0x7ff480069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4801598e0 0x7ff480069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.029+0000 7ff47dd9b700 1 -- 172.21.15.98:0/574883050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470009a20 con 0x7ff480159510 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff47dd9b700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff46800c840 tx=0x7ff468007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468005c00 con 0x7ff480159510 2023-10-06T01:40:39.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468005d60 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48015ed60 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468007300 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.030+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48015b6b0 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.031+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480064ad0 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.031+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff46801b400 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.031+0000 7ff4767fc700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 0x7ff4600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.032+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff468083fc0 con 0x7ff480159510 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.032+0000 7ff47e59c700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 0x7ff4600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.032+0000 7ff47e59c700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 0x7ff4600693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff47800ac00 tx=0x7ff478006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.036+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff468051410 con 0x7ff480159510 2023-10-06T01:40:39.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.219+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4800cfeb0 con 0x7ff480159510 2023-10-06T01:40:39.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.219+0000 7ff4767fc700 1 -- 172.21.15.98:0/574883050 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4680076f0 con 0x7ff480159510 2023-10-06T01:40:39.220 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 msgr2=0x7ff4600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 0x7ff4600693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff47800ac00 tx=0x7ff478006f90 comp rx=0 tx=0).stop 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 msgr2=0x7ff480169410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff46800c840 tx=0x7ff468007db0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 shutdown_connections 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff460066f30 0x7ff4600693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4801598e0 0x7ff480069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4800cbae0 0x7ff4800c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 --2- 172.21.15.98:0/574883050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff480159510 0x7ff480169410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 >> 172.21.15.98:0/574883050 conn(0x7ff480089e90 msgr2=0x7ff4801558f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:39.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 shutdown_connections 2023-10-06T01:40:39.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.221+0000 7ff484e4a700 1 -- 172.21.15.98:0/574883050 wait complete. 2023-10-06T01:40:39.234 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T01:40:39.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.348+0000 7f01487a4700 1 Processor -- start 2023-10-06T01:40:39.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.348+0000 7f01487a4700 1 -- start start 2023-10-06T01:40:39.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f01487a4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01401595a0 0x7f01400aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f01487a4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01400b0ca0 con 0x7f01400b07a0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f01487a4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01400b0f80 con 0x7f01400b49e0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f01487a4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01400b1260 con 0x7f01401595a0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f0146d41700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f01400b07a0 0x7f01400b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41950/0 (socket says 172.21.15.98:41950) 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f0146d41700 1 -- 172.21.15.98:0/3330290930 learned_addr learned my addr 172.21.15.98:0/3330290930 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f0146540700 1 --2- 172.21.15.98:0/3330290930 >> v2:172.21.15.184:3300/0 conn(0x7f01401595a0 0x7f01400aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003472078 0 0) 0x7f01400b0ca0 con 0x7f01400b07a0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.349+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f012c003570 con 0x7f01400b07a0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217948737 0 0) 0x7f012c003570 con 0x7f01400b07a0 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 >> v2:172.21.15.184:3300/0 conn(0x7f01401595a0 msgr2=0x7f01400aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:39.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 --2- 172.21.15.98:0/3330290930 >> v2:172.21.15.184:3300/0 conn(0x7f01401595a0 0x7f01400aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 >> v1:172.21.15.184:6789/0 conn(0x7f01400b49e0 legacy=0x7f01400aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400b1540 con 0x7f01400b07a0 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f01380017f0 con 0x7f01400b07a0 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 >> v1:172.21.15.98:6789/0 conn(0x7f01400b07a0 legacy=0x7f01400b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f012c003a60 0x7f012c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f012c004810 0x7f012c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f012c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01400b1260 con 0x7f012c0051c0 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c00c0d0 con 0x7f012c003a60 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c00c290 con 0x7f012c004810 2023-10-06T01:40:39.354 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f012c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146540700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f012c003a60 0x7f012c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0145d3f700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f012c004810 0x7f012c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 -- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f012c004810 msgr2=0x7f012c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f012c004810 0x7f012c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 -- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f012c003a60 msgr2=0x7f012c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f012c003a60 0x7f012c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 -- 172.21.15.98:0/3330290930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c003570 con 0x7f012c0051c0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 -- 172.21.15.98:0/3330290930 reap_dead start 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f0146d41700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f012c00ba60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f013800a930 tx=0x7f01380021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd7cfe6e3c6128f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380070d0 con 0x7f012c0051c0 2023-10-06T01:40:39.355 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138007230 con 0x7f012c0051c0 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.350+0000 7f014553e700 1 -- 172.21.15.98:0/3330290930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380075d0 con 0x7f012c0051c0 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 -- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 msgr2=0x7f012c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f012c00ba60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f013800a930 tx=0x7f01380021d0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 -- 172.21.15.98:0/3330290930 shutdown_connections 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 --2- 172.21.15.98:0/3330290930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f012c00ba60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 -- 172.21.15.98:0/3330290930 >> 172.21.15.98:0/3330290930 conn(0x7f0140099830 msgr2=0x7f014006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 -- 172.21.15.98:0/3330290930 shutdown_connections 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.351+0000 7f01487a4700 1 -- 172.21.15.98:0/3330290930 wait complete. 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.352+0000 7f01487a4700 1 Processor -- start 2023-10-06T01:40:39.356 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.352+0000 7f01487a4700 1 -- start start 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 0x7f0140156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01400ceba0 0x7f01400cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01400cf580 con 0x7f012c0051c0 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014009df30 con 0x7f01400ceba0 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f01487a4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014015dd60 con 0x7f014015bf60 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41076/0 (socket says 172.21.15.98:41076) 2023-10-06T01:40:39.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 -- 172.21.15.98:0/259854701 learned_addr learned my addr 172.21.15.98:0/259854701 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146d41700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01400ceba0 0x7f01400cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0145d3f700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 0x7f0140156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 -- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 msgr2=0x7f0140156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 0x7f0140156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 -- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01400ceba0 msgr2=0x7f01400cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01400ceba0 0x7f01400cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 -- 172.21.15.98:0/259854701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014015d430 con 0x7f012c0051c0 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0145d3f700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 0x7f0140156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.353+0000 7f0146540700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f013c011a40 tx=0x7f013c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c005c10 con 0x7f012c0051c0 2023-10-06T01:40:39.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c005d70 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014015d710 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00f960 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014009dc00 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.354+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0140064ad0 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.355+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f013c00fd90 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.355+0000 7f01367fc700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.355+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f013c084e10 con 0x7f012c0051c0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.355+0000 7f0145d3f700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.356+0000 7f0145d3f700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 0x7f01240693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f014009e110 tx=0x7f0130009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.359+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f013c052130 con 0x7f012c0051c0 2023-10-06T01:40:39.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.545+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f01400b1820 con 0x7f012c0051c0 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.545+0000 7f01367fc700 1 -- 172.21.15.98:0/259854701 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7f013c055750 con 0x7f012c0051c0 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2023-10-06T01:40:39.546 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": "rbd" 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 2, 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:40:39.547 INFO:teuthology.orchestra.run.smithi098.stdout:] 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 msgr2=0x7f01240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 0x7f01240693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f014009e110 tx=0x7f0130009040 comp rx=0 tx=0).stop 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 msgr2=0x7f014009d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f013c011a40 tx=0x7f013c009120 comp rx=0 tx=0).stop 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 shutdown_connections 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0124066f30 0x7f01240693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01400ceba0 0x7f01400cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f012c0051c0 0x7f014009d6c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 --2- 172.21.15.98:0/259854701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f014015bf60 0x7f0140156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 >> 172.21.15.98:0/259854701 conn(0x7f0140099830 msgr2=0x7f01400b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 shutdown_connections 2023-10-06T01:40:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.547+0000 7f01487a4700 1 -- 172.21.15.98:0/259854701 wait complete. 2023-10-06T01:40:39.559 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T01:40:39.560 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T01:40:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.677+0000 7f9308aea700 1 Processor -- start 2023-10-06T01:40:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.677+0000 7f9308aea700 1 -- start start 2023-10-06T01:40:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9308aea700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f930406ed20 0x7f93040b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9308aea700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93040aee20 0x7f93040af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9308aea700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93040b2d20 con 0x7f930406f0f0 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9308aea700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930406fb80 con 0x7f930406ed20 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9308aea700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93040b0450 con 0x7f93040aee20 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f930406ed20 0x7f93040b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f930406ed20 0x7f93040b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41080/0 (socket says 172.21.15.98:41080) 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 -- 172.21.15.98:0/176485476 learned_addr learned my addr 172.21.15.98:0/176485476 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f92f9d9b700 1 --1- 172.21.15.98:0/176485476 >> v1:172.21.15.98:6789/0 conn(0x7f930406f0f0 0x7f93040b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41962/0 (socket says 172.21.15.98:41962) 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9302d9d700 1 --2- 172.21.15.98:0/176485476 >> v2:172.21.15.184:3300/0 conn(0x7f93040aee20 0x7f93040af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 -- 172.21.15.98:0/176485476 >> v2:172.21.15.184:3300/0 conn(0x7f93040aee20 msgr2=0x7f93040af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 --2- 172.21.15.98:0/176485476 >> v2:172.21.15.184:3300/0 conn(0x7f93040aee20 0x7f93040af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 -- 172.21.15.98:0/176485476 >> v1:172.21.15.98:6789/0 conn(0x7f930406f0f0 legacy=0x7f93040b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f9301d9b700 1 -- 172.21.15.98:0/176485476 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243622685 0 0) 0x7f93040b2d20 con 0x7f930406f0f0 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.678+0000 7f930259c700 1 -- 172.21.15.98:0/176485476 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040af7c0 con 0x7f930406ed20 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f930259c700 1 --2- 172.21.15.98:0/176485476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406ed20 0x7f93040b23a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93041576c0 tx=0x7f92f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5622bbfcac6fecf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9301d9b700 1 -- 172.21.15.98:0/176485476 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400b860 con 0x7f930406ed20 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9301d9b700 1 -- 172.21.15.98:0/176485476 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f400b9c0 con 0x7f930406ed20 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9301d9b700 1 -- 172.21.15.98:0/176485476 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400bd60 con 0x7f930406ed20 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 -- 172.21.15.98:0/176485476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406ed20 msgr2=0x7f93040b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 --2- 172.21.15.98:0/176485476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406ed20 0x7f93040b23a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93041576c0 tx=0x7f92f4009250 comp rx=0 tx=0).stop 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 -- 172.21.15.98:0/176485476 shutdown_connections 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 --2- 172.21.15.98:0/176485476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406ed20 0x7f93040b23a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 --2- 172.21.15.98:0/176485476 >> v2:172.21.15.184:3300/0 conn(0x7f93040aee20 0x7f93040af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 -- 172.21.15.98:0/176485476 >> 172.21.15.98:0/176485476 conn(0x7f9304068330 msgr2=0x7f9304068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 -- 172.21.15.98:0/176485476 shutdown_connections 2023-10-06T01:40:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.679+0000 7f9308aea700 1 -- 172.21.15.98:0/176485476 wait complete. 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.680+0000 7f9308aea700 1 Processor -- start 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.680+0000 7f9308aea700 1 -- start start 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f930406ed20 0x7f93040c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 0x7f930416dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041420b0 con 0x7f930406f0f0 2023-10-06T01:40:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93040a8730 con 0x7f93040aee20 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f9308aea700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9304142300 con 0x7f930406ed20 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f92f9d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f92f9d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41092/0 (socket says 172.21.15.98:41092) 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f92f9d9b700 1 -- 172.21.15.98:0/2432786776 learned_addr learned my addr 172.21.15.98:0/2432786776 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.681+0000 7f930259c700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f930406ed20 0x7f93040c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f9302d9d700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 0x7f930416dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 -- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f930406ed20 msgr2=0x7f93040c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f930406ed20 0x7f93040c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 -- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 msgr2=0x7f930416dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:39.687 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 0x7f930416dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 -- 172.21.15.98:0/2432786776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40094f0 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f9302d9d700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 0x7f930416dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92f9d9b700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c5b0 tx=0x7f92ec001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec005a70 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec005bd0 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0070d0 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93041425e0 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.682+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9304144df0 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.683+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304142b00 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.683+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92ec007480 con 0x7f930406f0f0 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.684+0000 7f92faffd700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:39.688 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.684+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f92ec0839a0 con 0x7f930406f0f0 2023-10-06T01:40:39.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.684+0000 7f930259c700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:39.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.684+0000 7f930259c700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 0x7f92e40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93041576c0 tx=0x7f92f4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:39.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.688+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9304142b00 con 0x7f930406f0f0 2023-10-06T01:40:39.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:39.871+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9304142b00 con 0x7f930406f0f0 2023-10-06T01:40:40.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.449+0000 7f92faffd700 1 -- 172.21.15.98:0/2432786776 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9304142b00 con 0x7f930406f0f0 2023-10-06T01:40:40.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 msgr2=0x7f92e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 0x7f92e40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93041576c0 tx=0x7f92f4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:40.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 msgr2=0x7f930416d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c5b0 tx=0x7f92ec001040 comp rx=0 tx=0).stop 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 shutdown_connections 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f92e4066f30 0x7f92e40693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93040aee20 0x7f930416dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f930406ed20 0x7f93040c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 --2- 172.21.15.98:0/2432786776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f930406f0f0 0x7f930416d590 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c5b0 tx=0x7f92ec001040 comp rx=0 tx=0).stop 2023-10-06T01:40:40.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.453+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 >> 172.21.15.98:0/2432786776 conn(0x7f9304068330 msgr2=0x7f93040b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:40.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.454+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 shutdown_connections 2023-10-06T01:40:40.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.455+0000 7f9308aea700 1 -- 172.21.15.98:0/2432786776 wait complete. 2023-10-06T01:40:40.456 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2023-10-06T01:40:40.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T01:40:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.589+0000 7f0d0af42700 1 Processor -- start 2023-10-06T01:40:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 -- start start 2023-10-06T01:40:40.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d040a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d040af3c0 0x7f0d040675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d040aa580 con 0x7f0d040ac0c0 2023-10-06T01:40:40.592 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0415d3e0 con 0x7f0d040af3c0 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.590+0000 7f0d0af42700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040a9fc0 con 0x7f0d040ac490 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d040af3c0 0x7f0d040675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d09f40700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0d040ac0c0 0x7f0d0415d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41970/0 (socket says 172.21.15.98:41970) 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d09f40700 1 -- 172.21.15.98:0/3924251335 learned_addr learned my addr 172.21.15.98:0/3924251335 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 -- 172.21.15.98:0/3924251335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 msgr2=0x7f0d040a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 --2- 172.21.15.98:0/3924251335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d040a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 -- 172.21.15.98:0/3924251335 >> v1:172.21.15.98:6789/0 conn(0x7f0d040ac0c0 legacy=0x7f0d0415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 -- 172.21.15.98:0/3924251335 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040aa2a0 con 0x7f0d040af3c0 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d08f3e700 1 -- 172.21.15.98:0/3924251335 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208115121 0 0) 0x7f0d040aa580 con 0x7f0d040ac0c0 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d0a741700 1 --2- 172.21.15.98:0/3924251335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d040675f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098e0 tx=0x7f0cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cd88cb9c6d160b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:40.593 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d08f3e700 1 -- 172.21.15.98:0/3924251335 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc009e80 con 0x7f0d040af3c0 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d08f3e700 1 -- 172.21.15.98:0/3924251335 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc00b860 con 0x7f0d040af3c0 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.591+0000 7f0d08f3e700 1 -- 172.21.15.98:0/3924251335 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00bbe0 con 0x7f0d040af3c0 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 -- 172.21.15.98:0/3924251335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 msgr2=0x7f0d040675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 --2- 172.21.15.98:0/3924251335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d040675f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098e0 tx=0x7f0cfc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 -- 172.21.15.98:0/3924251335 shutdown_connections 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 --2- 172.21.15.98:0/3924251335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d040675f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 --2- 172.21.15.98:0/3924251335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d040a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 -- 172.21.15.98:0/3924251335 >> 172.21.15.98:0/3924251335 conn(0x7f0d04069180 msgr2=0x7f0d040aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 -- 172.21.15.98:0/3924251335 shutdown_connections 2023-10-06T01:40:40.594 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.592+0000 7f0d0af42700 1 -- 172.21.15.98:0/3924251335 wait complete. 2023-10-06T01:40:40.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.593+0000 7f0d0af42700 1 Processor -- start 2023-10-06T01:40:40.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.593+0000 7f0d0af42700 1 -- start start 2023-10-06T01:40:40.596 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d040ac0c0 0x7f0d040b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d04157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c5520 con 0x7f0d040af3c0 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04169380 con 0x7f0d040ac0c0 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0415bf40 con 0x7f0d040ac490 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41108/0 (socket says 172.21.15.98:41108) 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 -- 172.21.15.98:0/145442671 learned_addr learned my addr 172.21.15.98:0/145442671 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 -- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 msgr2=0x7f0d04157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d04157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 -- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d040ac0c0 msgr2=0x7f0d040b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:40.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d040ac0c0 0x7f0d040b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 -- 172.21.15.98:0/145442671 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009420 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0a741700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ce0f0 tx=0x7f0cfc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc01a640 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc01a7a0 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc01ab60 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.594+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0415c220 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.595+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0415c7e0 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.595+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0405f270 con 0x7f0d040af3c0 2023-10-06T01:40:40.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.596+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cfc011420 con 0x7f0d040af3c0 2023-10-06T01:40:40.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.596+0000 7f0cf9ffb700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 0x7f0ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:40.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.596+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f0cfc0865b0 con 0x7f0d040af3c0 2023-10-06T01:40:40.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.596+0000 7f0d09f40700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 0x7f0ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:40.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.597+0000 7f0d09f40700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 0x7f0ce80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d00001560 tx=0x7f0d000016c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:40.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.601+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cfc053760 con 0x7f0d040af3c0 2023-10-06T01:40:40.783 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:40.782+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0d0415d070 con 0x7f0d040af3c0 2023-10-06T01:40:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T01:40:41.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.454+0000 7f0cf9ffb700 1 -- 172.21.15.98:0/145442671 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0cfc056d80 con 0x7f0d040af3c0 2023-10-06T01:40:41.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.466+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 msgr2=0x7f0ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.466+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 0x7f0ce80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d00001560 tx=0x7f0d000016c0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.466+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 msgr2=0x7f0d04155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.466+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ce0f0 tx=0x7f0cfc017000 comp rx=0 tx=0).stop 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 shutdown_connections 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0ce8066f30 0x7f0ce80693e0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d00001560 tx=0x7f0d000016c0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d040af3c0 0x7f0d04155800 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d040ac0c0 0x7f0d040b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 --2- 172.21.15.98:0/145442671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d040ac490 0x7f0d04157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.467+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 >> 172.21.15.98:0/145442671 conn(0x7f0d04069180 msgr2=0x7f0d041585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:41.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.469+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 shutdown_connections 2023-10-06T01:40:41.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.469+0000 7f0d0af42700 1 -- 172.21.15.98:0/145442671 wait complete. 2023-10-06T01:40:41.471 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2023-10-06T01:40:41.484 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T01:40:41.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.601+0000 7f05d146c700 1 Processor -- start 2023-10-06T01:40:41.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.601+0000 7f05d146c700 1 -- start start 2023-10-06T01:40:41.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05d146c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f05cc0cd980 0x7f05cc159c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05d146c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0b5f30 con 0x7f05cc0b4a60 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05d146c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0b3640 con 0x7f05cc0b4eb0 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05d146c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc0b63c0 con 0x7f05cc0cd980 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05c27fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f05cc0b4a60 0x7f05cc0d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41984/0 (socket says 172.21.15.98:41984) 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.602+0000 7f05c27fc700 1 -- 172.21.15.98:0/513165780 learned_addr learned my addr 172.21.15.98:0/513165780 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05caffd700 1 --2- 172.21.15.98:0/513165780 >> v2:172.21.15.184:3300/0 conn(0x7f05cc0cd980 0x7f05cc159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05ca7fc700 1 -- 172.21.15.98:0/513165780 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241287469 0 0) 0x7f05cc0b5f30 con 0x7f05cc0b4a60 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05ca7fc700 1 -- 172.21.15.98:0/513165780 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b8003570 con 0x7f05cc0b4a60 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05caffd700 1 -- 172.21.15.98:0/513165780 >> v1:172.21.15.184:6789/0 conn(0x7f05cc0b4eb0 legacy=0x7f05cc163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.605 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05caffd700 1 -- 172.21.15.98:0/513165780 >> v1:172.21.15.98:6789/0 conn(0x7f05cc0b4a60 legacy=0x7f05cc0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05caffd700 1 -- 172.21.15.98:0/513165780 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc0b5320 con 0x7f05cc0cd980 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05caffd700 1 --2- 172.21.15.98:0/513165780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc159c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098e0 tx=0x7f05bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562f87e23c478dfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05ca7fc700 1 -- 172.21.15.98:0/513165780 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00b640 con 0x7f05cc0cd980 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05ca7fc700 1 -- 172.21.15.98:0/513165780 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc00b7a0 con 0x7f05cc0cd980 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.603+0000 7f05ca7fc700 1 -- 172.21.15.98:0/513165780 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00bb20 con 0x7f05cc0cd980 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 -- 172.21.15.98:0/513165780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 msgr2=0x7f05cc159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 --2- 172.21.15.98:0/513165780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc159c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098e0 tx=0x7f05bc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 -- 172.21.15.98:0/513165780 shutdown_connections 2023-10-06T01:40:41.606 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 --2- 172.21.15.98:0/513165780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc159c10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 -- 172.21.15.98:0/513165780 >> 172.21.15.98:0/513165780 conn(0x7f05cc06ed20 msgr2=0x7f05cc06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:41.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 -- 172.21.15.98:0/513165780 shutdown_connections 2023-10-06T01:40:41.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.604+0000 7f05d146c700 1 -- 172.21.15.98:0/513165780 wait complete. 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.605+0000 7f05d146c700 1 Processor -- start 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.605+0000 7f05d146c700 1 -- start start 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05cc0b4eb0 0x7f05cc0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:41.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15d230 con 0x7f05cc0b4a60 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15c620 con 0x7f05cc0cd980 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15c900 con 0x7f05cc0b4eb0 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41120/0 (socket says 172.21.15.98:41120) 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 -- 172.21.15.98:0/4252688323 learned_addr learned my addr 172.21.15.98:0/4252688323 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 -- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05cc0b4eb0 msgr2=0x7f05cc0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05cc0b4eb0 0x7f05cc0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 -- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 msgr2=0x7f05cc15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05cb7fe700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:41.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 -- 172.21.15.98:0/4252688323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009420 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05cb7fe700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05caffd700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0093f0 tx=0x7f05bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00bb20 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc011420 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc15cbe0 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.606+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc0117e0 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.607+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc13d3c0 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.607+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc063b60 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.608+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05bc011bd0 con 0x7f05cc0b4a60 2023-10-06T01:40:41.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.608+0000 7f05c37fe700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 0x7f05ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:41.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.608+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f05bc086270 con 0x7f05cc0b4a60 2023-10-06T01:40:41.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.608+0000 7f05c27fc700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 0x7f05ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:41.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.609+0000 7f05c27fc700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 0x7f05ac0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05b40058d0 tx=0x7f05b40079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:41.613 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.613+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05bc0532a0 con 0x7f05cc0b4a60 2023-10-06T01:40:41.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:41.843+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f05cc064ad0 con 0x7f05cc0b4a60 2023-10-06T01:40:42.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.468+0000 7f05c37fe700 1 -- 172.21.15.98:0/4252688323 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f05bc0568c0 con 0x7f05cc0b4a60 2023-10-06T01:40:42.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 msgr2=0x7f05ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 0x7f05ac0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05b40058d0 tx=0x7f05b40079d0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 msgr2=0x7f05cc13ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0093f0 tx=0x7f05bc017000 comp rx=0 tx=0).stop 2023-10-06T01:40:42.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 shutdown_connections 2023-10-06T01:40:42.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f05ac066f30 0x7f05ac0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05cc0cd980 0x7f05cc15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05cc0b4a60 0x7f05cc13ce80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 --2- 172.21.15.98:0/4252688323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05cc0b4eb0 0x7f05cc0cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.481+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 >> 172.21.15.98:0/4252688323 conn(0x7f05cc06ed20 msgr2=0x7f05cc063450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:42.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.485+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 shutdown_connections 2023-10-06T01:40:42.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.485+0000 7f05d146c700 1 -- 172.21.15.98:0/4252688323 wait complete. 2023-10-06T01:40:42.487 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 6 and data pool 7 2023-10-06T01:40:42.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.617+0000 7f774da49700 1 Processor -- start 2023-10-06T01:40:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.617+0000 7f774da49700 1 -- start start 2023-10-06T01:40:42.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774da49700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7748159480 0x7f77480b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774da49700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f77480b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774da49700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77480cb7e0 con 0x7f77480b3610 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774da49700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480b08a0 con 0x7f7748159480 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774da49700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480cb190 con 0x7f77480b2140 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7748159480 0x7f77480b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f773ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f77480b3610 0x7f77480cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45620/0 (socket says 172.21.15.98:45620) 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7748159480 0x7f77480b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46870/0 (socket says 172.21.15.98:46870) 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f773ffff700 1 -- 172.21.15.98:0/3065024913 learned_addr learned my addr 172.21.15.98:0/3065024913 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:42.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774d248700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f77480b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 -- 172.21.15.98:0/3065024913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 msgr2=0x7f77480b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f77480b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 -- 172.21.15.98:0/3065024913 >> v1:172.21.15.98:6789/0 conn(0x7f77480b3610 legacy=0x7f77480cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f774ca47700 1 -- 172.21.15.98:0/3065024913 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480cb470 con 0x7f7748159480 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.618+0000 7f773f7fe700 1 -- 172.21.15.98:0/3065024913 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819300451 0 0) 0x7f77480cb7e0 con 0x7f77480b3610 2023-10-06T01:40:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774ca47700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480b30d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f77440098e0 tx=0x7f77440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9456c30f4721a94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f773f7fe700 1 -- 172.21.15.98:0/3065024913 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744009e80 con 0x7f7748159480 2023-10-06T01:40:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f773f7fe700 1 -- 172.21.15.98:0/3065024913 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774400b860 con 0x7f7748159480 2023-10-06T01:40:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f773f7fe700 1 -- 172.21.15.98:0/3065024913 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774400bbe0 con 0x7f7748159480 2023-10-06T01:40:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 -- 172.21.15.98:0/3065024913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 msgr2=0x7f77480b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480b30d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f77440098e0 tx=0x7f77440091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 -- 172.21.15.98:0/3065024913 shutdown_connections 2023-10-06T01:40:42.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f77480b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 --2- 172.21.15.98:0/3065024913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480b30d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 -- 172.21.15.98:0/3065024913 >> 172.21.15.98:0/3065024913 conn(0x7f774806ed20 msgr2=0x7f774806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:42.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 -- 172.21.15.98:0/3065024913 shutdown_connections 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.619+0000 7f774da49700 1 -- 172.21.15.98:0/3065024913 wait complete. 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.620+0000 7f774da49700 1 Processor -- start 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 -- start start 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f7748155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 0x7f77480c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480c9190 con 0x7f7748159480 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480c9430 con 0x7f77480b3610 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774da49700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480c9710 con 0x7f77480b2140 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774d248700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774d248700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46872/0 (socket says 172.21.15.98:46872) 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774d248700 1 -- 172.21.15.98:0/1313208045 learned_addr learned my addr 172.21.15.98:0/1313208045 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:42.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.621+0000 7f774ca47700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f7748155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f773ffff700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 0x7f77480c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 -- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 msgr2=0x7f7748155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f7748155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 -- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 msgr2=0x7f77480c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 0x7f77480c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 -- 172.21.15.98:0/1313208045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744009420 con 0x7f7748159480 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f773ffff700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 0x7f77480c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774d248700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f774000ceb0 tx=0x7f7740007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000ae40 con 0x7f7748159480 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7740016430 con 0x7f7748159480 2023-10-06T01:40:42.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480c99f0 con 0x7f7748159480 2023-10-06T01:40:42.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400167d0 con 0x7f7748159480 2023-10-06T01:40:42.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.622+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7748155b80 con 0x7f7748159480 2023-10-06T01:40:42.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.623+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77480ca1f0 con 0x7f7748159480 2023-10-06T01:40:42.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.623+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7740003020 con 0x7f7748159480 2023-10-06T01:40:42.636 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.624+0000 7f773cff9700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 0x7f77340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.637 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.624+0000 7f774ca47700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 0x7f77340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.637 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.624+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7740012070 con 0x7f7748159480 2023-10-06T01:40:42.637 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.624+0000 7f774ca47700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 0x7f77340693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77440091e0 tx=0x7f7744017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.628+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7740052600 con 0x7f7748159480 2023-10-06T01:40:42.813 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.813+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77480c9f10 con 0x7f7748159480 2023-10-06T01:40:42.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.814+0000 7f773cff9700 1 -- 172.21.15.98:0/1313208045 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f7740055c20 con 0x7f7748159480 2023-10-06T01:40:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:42.815 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":35,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:40:41.844805+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 msgr2=0x7f77340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 0x7f77340693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77440091e0 tx=0x7f7744017000 comp rx=0 tx=0).stop 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 msgr2=0x7f77480c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f774000ceb0 tx=0x7f7740007870 comp rx=0 tx=0).stop 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 shutdown_connections 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7734066f30 0x7f77340693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7748159480 0x7f77480c62a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77480b2140 0x7f7748155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 --2- 172.21.15.98:0/1313208045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77480b3610 0x7f77480c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 >> 172.21.15.98:0/1313208045 conn(0x7f774806ed20 msgr2=0x7f7748156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:42.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 shutdown_connections 2023-10-06T01:40:42.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.816+0000 7f774da49700 1 -- 172.21.15.98:0/1313208045 wait complete. 2023-10-06T01:40:42.829 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.942+0000 7f59d1654700 1 Processor -- start 2023-10-06T01:40:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.942+0000 7f59d1654700 1 -- start start 2023-10-06T01:40:42.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59d1654700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f59cc1595a0 0x7f59cc0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59d1654700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f59cc0b49e0 0x7f59cc0aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59d1654700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc0b0ca0 con 0x7f59cc0b07a0 2023-10-06T01:40:42.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59d1654700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc0b0f80 con 0x7f59cc1595a0 2023-10-06T01:40:42.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59d1654700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc0b1260 con 0x7f59cc0b49e0 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59caffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f59cc1595a0 0x7f59cc0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59cb7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f59cc0b07a0 0x7f59cc0b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45626/0 (socket says 172.21.15.98:45626) 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59cb7fe700 1 -- 172.21.15.98:0/320505082 learned_addr learned my addr 172.21.15.98:0/320505082 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59c27fc700 1 --2- 172.21.15.98:0/320505082 >> v2:172.21.15.184:3300/0 conn(0x7f59cc0b49e0 0x7f59cc0aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59caffd700 1 -- 172.21.15.98:0/320505082 >> v2:172.21.15.184:3300/0 conn(0x7f59cc0b49e0 msgr2=0x7f59cc0aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59caffd700 1 --2- 172.21.15.98:0/320505082 >> v2:172.21.15.184:3300/0 conn(0x7f59cc0b49e0 0x7f59cc0aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59caffd700 1 -- 172.21.15.98:0/320505082 >> v1:172.21.15.98:6789/0 conn(0x7f59cc0b07a0 legacy=0x7f59cc0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59caffd700 1 -- 172.21.15.98:0/320505082 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0b1540 con 0x7f59cc1595a0 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.943+0000 7f59ca7fc700 1 -- 172.21.15.98:0/320505082 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288303485 0 0) 0x7f59cc0b0ca0 con 0x7f59cc0b07a0 2023-10-06T01:40:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59caffd700 1 --2- 172.21.15.98:0/320505082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc0aa570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098d0 tx=0x7f59bc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce6ebcc856a2de9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59ca7fc700 1 -- 172.21.15.98:0/320505082 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc009e70 con 0x7f59cc1595a0 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59ca7fc700 1 -- 172.21.15.98:0/320505082 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc005c10 con 0x7f59cc1595a0 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59ca7fc700 1 -- 172.21.15.98:0/320505082 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00b610 con 0x7f59cc1595a0 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 -- 172.21.15.98:0/320505082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 msgr2=0x7f59cc0aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 --2- 172.21.15.98:0/320505082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc0aa570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098d0 tx=0x7f59bc009120 comp rx=0 tx=0).stop 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 -- 172.21.15.98:0/320505082 shutdown_connections 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 --2- 172.21.15.98:0/320505082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc0aa570 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 --2- 172.21.15.98:0/320505082 >> v2:172.21.15.184:3300/0 conn(0x7f59cc0b49e0 0x7f59cc0aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 -- 172.21.15.98:0/320505082 >> 172.21.15.98:0/320505082 conn(0x7f59cc099830 msgr2=0x7f59cc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 -- 172.21.15.98:0/320505082 shutdown_connections 2023-10-06T01:40:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.944+0000 7f59d1654700 1 -- 172.21.15.98:0/320505082 wait complete. 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.945+0000 7f59d1654700 1 Processor -- start 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 -- start start 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59cc0b07a0 0x7f59cc16a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59cc0b49e0 0x7f59cc156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc09aa20 con 0x7f59cc1595a0 2023-10-06T01:40:42.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc09a130 con 0x7f59cc0b07a0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59d1654700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc157010 con 0x7f59cc0b49e0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59cb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59cb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46890/0 (socket says 172.21.15.98:46890) 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.946+0000 7f59cb7fe700 1 -- 172.21.15.98:0/2515306869 learned_addr learned my addr 172.21.15.98:0/2515306869 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59caffd700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59cc0b07a0 0x7f59cc16a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59c27fc700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59cc0b49e0 0x7f59cc156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 -- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59cc0b49e0 msgr2=0x7f59cc156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59cc0b49e0 0x7f59cc156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 -- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59cc0b07a0 msgr2=0x7f59cc16a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59cc0b07a0 0x7f59cc16a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 -- 172.21.15.98:0/2515306869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0093c0 con 0x7f59cc1595a0 2023-10-06T01:40:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59cb7fe700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cc90 tx=0x7f59c4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400cea0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c400ab70 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40070f0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc09a3d0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.947+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc09dff0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.948+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc09ab90 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.948+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59c40074a0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.949+0000 7f59c37fe700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 0x7f59ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.949+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f59c4084cb0 con 0x7f59cc1595a0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.949+0000 7f59caffd700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 0x7f59ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.949+0000 7f59caffd700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 0x7f59ac0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009160 tx=0x7f59bc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:42.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:42.953+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59c4051d00 con 0x7f59cc1595a0 2023-10-06T01:40:43.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.137+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59cc064ad0 con 0x7f59cc1595a0 2023-10-06T01:40:43.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.138+0000 7f59c37fe700 1 -- 172.21.15.98:0/2515306869 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f59c4055320 con 0x7f59cc1595a0 2023-10-06T01:40:43.139 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:43.139 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":35,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:40:41.844805+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:43.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 msgr2=0x7f59ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:43.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 0x7f59ac0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009160 tx=0x7f59bc016000 comp rx=0 tx=0).stop 2023-10-06T01:40:43.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 msgr2=0x7f59cc09d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:43.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cc90 tx=0x7f59c4001a80 comp rx=0 tx=0).stop 2023-10-06T01:40:43.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 shutdown_connections 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f59ac066f30 0x7f59ac0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59cc1595a0 0x7f59cc09d730 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59cc0b07a0 0x7f59cc16a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 --2- 172.21.15.98:0/2515306869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59cc0b49e0 0x7f59cc156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 >> 172.21.15.98:0/2515306869 conn(0x7f59cc099830 msgr2=0x7f59cc068400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 shutdown_connections 2023-10-06T01:40:43.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.140+0000 7f59d1654700 1 -- 172.21.15.98:0/2515306869 wait complete. 2023-10-06T01:40:43.153 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T01:40:43.270 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.269+0000 7f4b06eac700 1 Processor -- start 2023-10-06T01:40:43.270 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.269+0000 7f4b06eac700 1 -- start start 2023-10-06T01:40:43.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4b06eac700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b00159340 0x7f4b000af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4b06eac700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b00159710 0x7f4b000cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4b06eac700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b000cbed0 con 0x7f4b000cba70 2023-10-06T01:40:43.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4b06eac700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000cf820 con 0x7f4b00159710 2023-10-06T01:40:43.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4b06eac700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000cfb00 con 0x7f4b00159340 2023-10-06T01:40:43.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.270+0000 7f4af7fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b00159710 0x7f4b000cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b00159710 0x7f4b000cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46896/0 (socket says 172.21.15.98:46896) 2023-10-06T01:40:43.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 -- 172.21.15.98:0/380639831 learned_addr learned my addr 172.21.15.98:0/380639831 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:43.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4b04c48700 1 --2- 172.21.15.98:0/380639831 >> v2:172.21.15.184:3300/0 conn(0x7f4b00159340 0x7f4b000af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 -- 172.21.15.98:0/380639831 >> v2:172.21.15.184:3300/0 conn(0x7f4b00159340 msgr2=0x7f4b000af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 --2- 172.21.15.98:0/380639831 >> v2:172.21.15.184:3300/0 conn(0x7f4b00159340 0x7f4b000af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 -- 172.21.15.98:0/380639831 >> v1:172.21.15.184:6789/0 conn(0x7f4b000cba70 legacy=0x7f4b000cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 -- 172.21.15.98:0/380639831 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000cfde0 con 0x7f4b00159710 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af7fff700 1 --2- 172.21.15.98:0/380639831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159710 0x7f4b000cb530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4b00071290 tx=0x7f4af0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6e7541ddf16924 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af77fe700 1 -- 172.21.15.98:0/380639831 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000b820 con 0x7f4b00159710 2023-10-06T01:40:43.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af77fe700 1 -- 172.21.15.98:0/380639831 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000b980 con 0x7f4b00159710 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.271+0000 7f4af77fe700 1 -- 172.21.15.98:0/380639831 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000bd20 con 0x7f4b00159710 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 -- 172.21.15.98:0/380639831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159710 msgr2=0x7f4b000cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 --2- 172.21.15.98:0/380639831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159710 0x7f4b000cb530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4b00071290 tx=0x7f4af0009250 comp rx=0 tx=0).stop 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 -- 172.21.15.98:0/380639831 shutdown_connections 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 --2- 172.21.15.98:0/380639831 >> v2:172.21.15.184:3300/0 conn(0x7f4b00159340 0x7f4b000af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 --2- 172.21.15.98:0/380639831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159710 0x7f4b000cb530 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 -- 172.21.15.98:0/380639831 >> 172.21.15.98:0/380639831 conn(0x7f4b00099720 msgr2=0x7f4b0006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 -- 172.21.15.98:0/380639831 shutdown_connections 2023-10-06T01:40:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.272+0000 7f4b06eac700 1 -- 172.21.15.98:0/380639831 wait complete. 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.273+0000 7f4b06eac700 1 Processor -- start 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.273+0000 7f4b06eac700 1 -- start start 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 0x7f4b0014b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b00159710 0x7f4b000c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0014b570 con 0x7f4b00159340 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0016ad40 con 0x7f4b000cba70 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00169e70 con 0x7f4b00159710 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46908/0 (socket says 172.21.15.98:46908) 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 -- 172.21.15.98:0/3504279619 learned_addr learned my addr 172.21.15.98:0/3504279619 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b04c48700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 0x7f4b0014b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b05449700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b00159710 0x7f4b000c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 -- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b00159710 msgr2=0x7f4b000c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b00159710 0x7f4b000c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 -- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 msgr2=0x7f4b0014b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 0x7f4b0014b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 -- 172.21.15.98:0/3504279619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00094f0 con 0x7f4b00159340 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b04c48700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 0x7f4b0014b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af7fff700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4af00030c0 tx=0x7f4af0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0011e60 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af001b680 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0016a130 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af001ba20 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.274+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0016a6f0 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.275+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00064ad0 con 0x7f4b00159340 2023-10-06T01:40:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.276+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4af001c450 con 0x7f4b00159340 2023-10-06T01:40:43.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.276+0000 7f4af4ff9700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 0x7f4aec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:43.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.276+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4af00862b0 con 0x7f4b00159340 2023-10-06T01:40:43.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.276+0000 7f4b04c48700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 0x7f4aec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:43.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.277+0000 7f4b04c48700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 0x7f4aec0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ca590 tx=0x7f4afc007760 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:43.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.280+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4af0053280 con 0x7f4b00159340 2023-10-06T01:40:43.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:43.508+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b000cbed0 con 0x7f4b00159340 2023-10-06T01:40:44.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.480+0000 7f4af4ff9700 1 -- 172.21.15.98:0/3504279619 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b000cbed0 con 0x7f4b00159340 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 msgr2=0x7f4aec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 0x7f4aec0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ca590 tx=0x7f4afc007760 comp rx=0 tx=0).stop 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 msgr2=0x7f4b000c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4af00030c0 tx=0x7f4af0016000 comp rx=0 tx=0).stop 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 shutdown_connections 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4aec066f30 0x7f4aec0693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b00159710 0x7f4b000c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b000cba70 0x7f4b0014b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 --2- 172.21.15.98:0/3504279619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b00159340 0x7f4b000c96e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 >> 172.21.15.98:0/3504279619 conn(0x7f4b00099720 msgr2=0x7f4b00069030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:44.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 shutdown_connections 2023-10-06T01:40:44.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.483+0000 7f4b06eac700 1 -- 172.21.15.98:0/3504279619 wait complete. 2023-10-06T01:40:44.498 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T01:40:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.621+0000 7f11ce52f700 1 Processor -- start 2023-10-06T01:40:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.621+0000 7f11ce52f700 1 -- start start 2023-10-06T01:40:44.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ce52f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 0x7f11c80c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ce52f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11c80c93f0 0x7f11c80c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ce52f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80cae10 con 0x7f11c80b0610 2023-10-06T01:40:44.624 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ce52f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c8157810 con 0x7f11c80c93f0 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ce52f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c80c9b20 con 0x7f11c80cce60 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cdd2e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11c80c93f0 0x7f11c80c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cd52d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f11c80b0610 0x7f11c80cc750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45630/0 (socket says 172.21.15.98:45630) 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cd52d700 1 -- 172.21.15.98:0/1888127591 learned_addr learned my addr 172.21.15.98:0/1888127591 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11ccd2c700 1 --2- 172.21.15.98:0/1888127591 >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 0x7f11c80c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cdd2e700 1 -- 172.21.15.98:0/1888127591 >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 msgr2=0x7f11c80c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cdd2e700 1 --2- 172.21.15.98:0/1888127591 >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 0x7f11c80c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cdd2e700 1 -- 172.21.15.98:0/1888127591 >> v1:172.21.15.98:6789/0 conn(0x7f11c80b0610 legacy=0x7f11c80cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11cdd2e700 1 -- 172.21.15.98:0/1888127591 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80c9e00 con 0x7f11c80c93f0 2023-10-06T01:40:44.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.622+0000 7f11bffff700 1 -- 172.21.15.98:0/1888127591 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217469839 0 0) 0x7f11c80cae10 con 0x7f11c80b0610 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ccd2c700 1 --2- 172.21.15.98:0/1888127591 >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 0x7f11c80c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11cdd2e700 1 --2- 172.21.15.98:0/1888127591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80c2f80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f11c80b4170 tx=0x7f11c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e37f9e9213c921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11bffff700 1 -- 172.21.15.98:0/1888127591 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000e070 con 0x7f11c80c93f0 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11bffff700 1 -- 172.21.15.98:0/1888127591 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c000bc30 con 0x7f11c80c93f0 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11bffff700 1 -- 172.21.15.98:0/1888127591 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0012430 con 0x7f11c80c93f0 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 -- 172.21.15.98:0/1888127591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 msgr2=0x7f11c80c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 --2- 172.21.15.98:0/1888127591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80c2f80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f11c80b4170 tx=0x7f11c0009780 comp rx=0 tx=0).stop 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 -- 172.21.15.98:0/1888127591 shutdown_connections 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 --2- 172.21.15.98:0/1888127591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80c2f80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 --2- 172.21.15.98:0/1888127591 >> v2:172.21.15.184:3300/0 conn(0x7f11c80cce60 0x7f11c80c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 -- 172.21.15.98:0/1888127591 >> 172.21.15.98:0/1888127591 conn(0x7f11c8096a70 msgr2=0x7f11c8096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:44.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 -- 172.21.15.98:0/1888127591 shutdown_connections 2023-10-06T01:40:44.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.623+0000 7f11ce52f700 1 -- 172.21.15.98:0/1888127591 wait complete. 2023-10-06T01:40:44.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.624+0000 7f11ce52f700 1 Processor -- start 2023-10-06T01:40:44.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 -- start start 2023-10-06T01:40:44.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11c80b0610 0x7f11c80b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11c80cce60 0x7f11c81512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b3dc0 con 0x7f11c80c93f0 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b0f10 con 0x7f11c80cce60 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ce52f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b17b0 con 0x7f11c80b0610 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ccd2c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.625+0000 7f11ccd2c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46932/0 (socket says 172.21.15.98:46932) 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 -- 172.21.15.98:0/449371668 learned_addr learned my addr 172.21.15.98:0/449371668 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11cd52d700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11c80b0610 0x7f11c80b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11cdd2e700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11c80cce60 0x7f11c81512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 -- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11c80b0610 msgr2=0x7f11c80b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11c80b0610 0x7f11c80b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 -- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11c80cce60 msgr2=0x7f11c81512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11c80cce60 0x7f11c81512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 -- 172.21.15.98:0/449371668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009a20 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ccd2c700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c850 tx=0x7f11b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005a00 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8005b60 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80c7f80 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8011600 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.626+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80b1440 con 0x7f11c80c93f0 2023-10-06T01:40:44.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.627+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8152d10 con 0x7f11c80c93f0 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.628+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11b80119f0 con 0x7f11c80c93f0 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.628+0000 7f11bd7fa700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 0x7f11ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.628+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f11b8084420 con 0x7f11c80c93f0 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.628+0000 7f11cd52d700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 0x7f11ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.628+0000 7f11cd52d700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 0x7f11ac0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011e0 tx=0x7f11c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:44.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.632+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11b80513f0 con 0x7f11c80c93f0 2023-10-06T01:40:44.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:44.858+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f11c805f270 con 0x7f11c80c93f0 2023-10-06T01:40:45.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.483+0000 7f11bd7fa700 1 -- 172.21.15.98:0/449371668 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11b8054a10 con 0x7f11c80c93f0 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 msgr2=0x7f11ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 0x7f11ac0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011e0 tx=0x7f11c4009040 comp rx=0 tx=0).stop 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 msgr2=0x7f11c80b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c850 tx=0x7f11b8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 shutdown_connections 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f11ac066f30 0x7f11ac0693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11c80cce60 0x7f11c81512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11c80b0610 0x7f11c80b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 --2- 172.21.15.98:0/449371668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c80c93f0 0x7f11c80b3880 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 >> 172.21.15.98:0/449371668 conn(0x7f11c8096a70 msgr2=0x7f11c806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 shutdown_connections 2023-10-06T01:40:45.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.486+0000 7f11ce52f700 1 -- 172.21.15.98:0/449371668 wait complete. 2023-10-06T01:40:45.500 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:45.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.614+0000 7f342b64b700 1 Processor -- start 2023-10-06T01:40:45.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.614+0000 7f342b64b700 1 -- start start 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f342b64b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f34240b29f0 0x7f3424159160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f342b64b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34240cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f342b64b700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240b0de0 con 0x7f34240b2620 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f34293e7700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f34240b2620 0x7f34240b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45636/0 (socket says 172.21.15.98:45636) 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f34293e7700 1 -- 172.21.15.98:0/4278562931 learned_addr learned my addr 172.21.15.98:0/4278562931 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f342b64b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240cbfe0 con 0x7f34240b29f0 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.615+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cb3d0 con 0x7f34241596a0 2023-10-06T01:40:45.619 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f3428be6700 1 --2- 172.21.15.98:0/4278562931 >> v2:172.21.15.184:3300/0 conn(0x7f34240b29f0 0x7f3424159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088251786 0 0) 0x7f3420004f00 con 0x7f34240b2620 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f3429be8700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34240cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003570 con 0x7f34240b2620 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273245264 0 0) 0x7f3410003570 con 0x7f34240b2620 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 msgr2=0x7f34240cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34240cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 >> v2:172.21.15.184:3300/0 conn(0x7f34240b29f0 msgr2=0x7f3424159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 --2- 172.21.15.98:0/4278562931 >> v2:172.21.15.184:3300/0 conn(0x7f34240b29f0 0x7f3424159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240cb6b0 con 0x7f34240b2620 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f34200053f0 con 0x7f34240b2620 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 >> v1:172.21.15.98:6789/0 conn(0x7f34240b2620 legacy=0x7f34240b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f3410003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3410004810 0x7f3410004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cbfe0 con 0x7f3410003a60 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cb3d0 con 0x7f3410004810 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341000ba60 con 0x7f34241596a0 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f3410003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f3428be6700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3410004810 0x7f3410004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f3429be8700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 -- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 msgr2=0x7f34100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34241596a0 0x7f34100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 -- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3410004810 msgr2=0x7f3410004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:45.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3410004810 0x7f3410004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 -- 172.21.15.98:0/4278562931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410003570 con 0x7f3410003a60 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f3428be6700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3410004810 0x7f3410004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.616+0000 7f34293e7700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f3410003e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f342000ab30 tx=0x7f342000a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930309140c883e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000c590 con 0x7f3410003a60 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342000c6f0 con 0x7f3410003a60 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f341bfff700 1 -- 172.21.15.98:0/4278562931 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000cab0 con 0x7f3410003a60 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 msgr2=0x7f3410003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 --2- 172.21.15.98:0/4278562931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f3410003e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f342000ab30 tx=0x7f342000a1e0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f3429be8700 1 -- 172.21.15.98:0/4278562931 reap_dead start 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 shutdown_connections 2023-10-06T01:40:45.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 >> 172.21.15.98:0/4278562931 conn(0x7f342406ed20 msgr2=0x7f342406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:45.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 shutdown_connections 2023-10-06T01:40:45.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.617+0000 7f342b64b700 1 -- 172.21.15.98:0/4278562931 wait complete. 2023-10-06T01:40:45.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.618+0000 7f342b64b700 1 Processor -- start 2023-10-06T01:40:45.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.618+0000 7f342b64b700 1 -- start start 2023-10-06T01:40:45.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34241520b0 0x7f34240c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34240d0640 0x7f34240d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240c6f60 con 0x7f3410003a60 2023-10-06T01:40:45.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240d0ff0 con 0x7f34241520b0 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f342b64b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342416ab00 con 0x7f34240d0640 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f34293e7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f34293e7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46944/0 (socket says 172.21.15.98:46944) 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.619+0000 7f34293e7700 1 -- 172.21.15.98:0/3695268463 learned_addr learned my addr 172.21.15.98:0/3695268463 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f3429be8700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34240d0640 0x7f34240d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f3428be6700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34241520b0 0x7f34240c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 -- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34240d0640 msgr2=0x7f34240d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34240d0640 0x7f34240d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 -- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34241520b0 msgr2=0x7f34240c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34241520b0 0x7f34240c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 -- 172.21.15.98:0/3695268463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342416a1d0 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34293e7700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3420005c50 tx=0x7f3420016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000ca70 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342001e440 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342416a4b0 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342001e5f0 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.620+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3424169ee0 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.621+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3424064ad0 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.621+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3420025020 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.622+0000 7f34197fa700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.622+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3420086860 con 0x7f3410003a60 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.622+0000 7f3428be6700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.622+0000 7f3428be6700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 0x7f34080693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3414006dc0 tx=0x7f3414008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.626+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3420053830 con 0x7f3410003a60 2023-10-06T01:40:45.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.851+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34240b0de0 con 0x7f3410003a60 2023-10-06T01:40:45.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.851+0000 7f34197fa700 1 -- 172.21.15.98:0/3695268463 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f34240b0de0 con 0x7f3410003a60 2023-10-06T01:40:45.852 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:45.853 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/388888775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388888775},{"type":"v1","addr":"172.21.15.98:6836","nonce":388888775}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/582727435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":582727435},{"type":"v1","addr":"172.21.15.98:6837","nonce":582727435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2425522185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2425522185},{"type":"v1","addr":"172.21.15.184:6834","nonce":2425522185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:45.479675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1929325397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1929325397},{"type":"v1","addr":"172.21.15.184:6835","nonce":1929325397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T01:40:45.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 msgr2=0x7f34080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 0x7f34080693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3414006dc0 tx=0x7f3414008040 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 msgr2=0x7f34240c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3420005c50 tx=0x7f3420016000 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 shutdown_connections 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34240d0640 0x7f34240d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3410003a60 0x7f34240c8c20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 --2- 172.21.15.98:0/3695268463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34241520b0 0x7f34240c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 >> 172.21.15.98:0/3695268463 conn(0x7f342406ed20 msgr2=0x7f34240b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 shutdown_connections 2023-10-06T01:40:45.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.854+0000 7f342b64b700 1 -- 172.21.15.98:0/3695268463 wait complete. 2023-10-06T01:40:45.856 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2023-10-06T01:40:45.866 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:40:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.979+0000 7f2492343700 1 Processor -- start 2023-10-06T01:40:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.979+0000 7f2492343700 1 -- start start 2023-10-06T01:40:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2492343700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2492343700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2492343700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0cb7e0 con 0x7f248c0cc1b0 2023-10-06T01:40:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2492343700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0b08a0 con 0x7f248c0b3610 2023-10-06T01:40:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2492343700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0cb190 con 0x7f248c159480 2023-10-06T01:40:45.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2490b40700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2491341700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2490b40700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:47062/0 (socket says 172.21.15.98:47062) 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2491341700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:57470/0 (socket says 172.21.15.98:57470) 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2490b40700 1 -- 172.21.15.98:0/1351649018 learned_addr learned my addr 172.21.15.98:0/1351649018 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.980+0000 7f2491b42700 1 --1- 172.21.15.98:0/1351649018 >> v1:172.21.15.184:6789/0 conn(0x7f248c0cc1b0 0x7f248c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:40850/0 (socket says 172.21.15.98:40850) 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2491341700 1 -- 172.21.15.98:0/1351649018 >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 msgr2=0x7f248c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2491341700 1 --2- 172.21.15.98:0/1351649018 >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2491341700 1 -- 172.21.15.98:0/1351649018 >> v1:172.21.15.184:6789/0 conn(0x7f248c0cc1b0 legacy=0x7f248c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2491341700 1 -- 172.21.15.98:0/1351649018 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cb470 con 0x7f248c159480 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2490b40700 1 --2- 172.21.15.98:0/1351649018 >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:45.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2491341700 1 --2- 172.21.15.98:0/1351649018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24880098e0 tx=0x7f24880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc588dfb42438b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2483fff700 1 -- 172.21.15.98:0/1351649018 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800b640 con 0x7f248c159480 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2483fff700 1 -- 172.21.15.98:0/1351649018 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800b7a0 con 0x7f248c159480 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2483fff700 1 -- 172.21.15.98:0/1351649018 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800bb40 con 0x7f248c159480 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2492343700 1 -- 172.21.15.98:0/1351649018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 msgr2=0x7f248c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2492343700 1 --2- 172.21.15.98:0/1351649018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24880098e0 tx=0x7f24880091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2492343700 1 -- 172.21.15.98:0/1351649018 shutdown_connections 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2492343700 1 --2- 172.21.15.98:0/1351649018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0b30d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.981+0000 7f2492343700 1 --2- 172.21.15.98:0/1351649018 >> v2:172.21.15.184:3300/0 conn(0x7f248c0b3610 0x7f248c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.982+0000 7f2492343700 1 -- 172.21.15.98:0/1351649018 >> 172.21.15.98:0/1351649018 conn(0x7f248c06ed20 msgr2=0x7f248c06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.982+0000 7f2492343700 1 -- 172.21.15.98:0/1351649018 shutdown_connections 2023-10-06T01:40:45.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.982+0000 7f2492343700 1 -- 172.21.15.98:0/1351649018 wait complete. 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.983+0000 7f2492343700 1 Processor -- start 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.983+0000 7f2492343700 1 -- start start 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 0x7f248c14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c5760 con 0x7f248c0cc1b0 2023-10-06T01:40:45.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c52d0 con 0x7f248c0b3610 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c46c0 con 0x7f248c159480 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46956/0 (socket says 172.21.15.98:46956) 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 -- 172.21.15.98:0/1743735391 learned_addr learned my addr 172.21.15.98:0/1743735391 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2491341700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 0x7f248c14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2491b42700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 -- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 msgr2=0x7f248c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 -- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 msgr2=0x7f248c14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 0x7f248c14e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 -- 172.21.15.98:0/1743735391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0094f0 con 0x7f248c0cc1b0 2023-10-06T01:40:45.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2491341700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 0x7f248c14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2490b40700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f247c00ec50 tx=0x7f247c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00c9d0 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c00cb30 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2488009420 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c013470 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.984+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c0c4c80 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.985+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c0c5440 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.986+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f247c013820 con 0x7f248c0cc1b0 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.986+0000 7f24817fa700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:45.989 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.986+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f247c084fe0 con 0x7f248c0cc1b0 2023-10-06T01:40:45.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.986+0000 7f2491341700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:45.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.986+0000 7f2491341700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2488003080 tx=0x7f2488017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:45.990 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:45.990+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f247c051fb0 con 0x7f248c0cc1b0 2023-10-06T01:40:46.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.175+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f248c159850 con 0x7f248c0cc1b0 2023-10-06T01:40:46.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.176+0000 7f24817fa700 1 -- 172.21.15.98:0/1743735391 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17167 (secure 0 0 0) 0x7f247c0555d0 con 0x7f248c0cc1b0 2023-10-06T01:40:46.177 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:46.177 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":36,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:40:43.465741+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 msgr2=0x7f24700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2488003080 tx=0x7f2488017000 comp rx=0 tx=0).stop 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 msgr2=0x7f248c0c3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f247c00ec50 tx=0x7f247c009250 comp rx=0 tx=0).stop 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 shutdown_connections 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f248c159480 0x7f248c0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c0b3610 0x7f248c14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 --2- 172.21.15.98:0/1743735391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f248c0cc1b0 0x7f248c0c3c40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 >> 172.21.15.98:0/1743735391 conn(0x7f248c06ed20 msgr2=0x7f248c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 shutdown_connections 2023-10-06T01:40:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.178+0000 7f2492343700 1 -- 172.21.15.98:0/1743735391 wait complete. 2023-10-06T01:40:46.190 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T01:40:46.191 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:46.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.305+0000 7f78b3428700 1 Processor -- start 2023-10-06T01:40:46.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.305+0000 7f78b3428700 1 -- start start 2023-10-06T01:40:46.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b3428700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b3428700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 0x7f78ac0ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b3428700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0cdb80 con 0x7f78ac15ac80 2023-10-06T01:40:46.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b3428700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac0ccf70 con 0x7f78ac15a8b0 2023-10-06T01:40:46.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b3428700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac0cd250 con 0x7f78ac0c8f60 2023-10-06T01:40:46.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b11c4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b11c4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46958/0 (socket says 172.21.15.98:46958) 2023-10-06T01:40:46.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b11c4700 1 -- 172.21.15.98:0/705894815 learned_addr learned my addr 172.21.15.98:0/705894815 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.306+0000 7f78b19c5700 1 --2- 172.21.15.98:0/705894815 >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 0x7f78ac0ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b11c4700 1 -- 172.21.15.98:0/705894815 >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 msgr2=0x7f78ac0ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b11c4700 1 --2- 172.21.15.98:0/705894815 >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 0x7f78ac0ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b11c4700 1 -- 172.21.15.98:0/705894815 >> v1:172.21.15.184:6789/0 conn(0x7f78ac15ac80 legacy=0x7f78ac0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b11c4700 1 -- 172.21.15.98:0/705894815 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0cd530 con 0x7f78ac15a8b0 2023-10-06T01:40:46.310 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b19c5700 1 --2- 172.21.15.98:0/705894815 >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 0x7f78ac0ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:46.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b11c4700 1 --2- 172.21.15.98:0/705894815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f78ac06e750 tx=0x7f78a8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f17b1471c923d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78a3fff700 1 -- 172.21.15.98:0/705894815 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8005c10 con 0x7f78ac15a8b0 2023-10-06T01:40:46.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78a3fff700 1 -- 172.21.15.98:0/705894815 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8005d70 con 0x7f78ac15a8b0 2023-10-06T01:40:46.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78a3fff700 1 -- 172.21.15.98:0/705894815 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800b880 con 0x7f78ac15a8b0 2023-10-06T01:40:46.311 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 -- 172.21.15.98:0/705894815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 msgr2=0x7f78ac0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 --2- 172.21.15.98:0/705894815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f78ac06e750 tx=0x7f78a8009220 comp rx=0 tx=0).stop 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 -- 172.21.15.98:0/705894815 shutdown_connections 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 --2- 172.21.15.98:0/705894815 >> v2:172.21.15.184:3300/0 conn(0x7f78ac0c8f60 0x7f78ac0ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 --2- 172.21.15.98:0/705894815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac0ce9c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.307+0000 7f78b3428700 1 -- 172.21.15.98:0/705894815 >> 172.21.15.98:0/705894815 conn(0x7f78ac067dd0 msgr2=0x7f78ac0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.308+0000 7f78b3428700 1 -- 172.21.15.98:0/705894815 shutdown_connections 2023-10-06T01:40:46.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.308+0000 7f78b3428700 1 -- 172.21.15.98:0/705894815 wait complete. 2023-10-06T01:40:46.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.309+0000 7f78b3428700 1 Processor -- start 2023-10-06T01:40:46.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.309+0000 7f78b3428700 1 -- start start 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.309+0000 7f78b3428700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 0x7f78ac167520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac16b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac15ac80 0x7f78ac16b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0acf40 con 0x7f78ac15a8b0 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0ac330 con 0x7f78ac0c8f60 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0ac610 con 0x7f78ac15ac80 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b11c4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 0x7f78ac167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac16b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b11c4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 0x7f78ac167520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:47084/0 (socket says 172.21.15.98:47084) 2023-10-06T01:40:46.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b11c4700 1 -- 172.21.15.98:0/3366878886 learned_addr learned my addr 172.21.15.98:0/3366878886 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b19c5700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac15ac80 0x7f78ac16b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 -- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac15ac80 msgr2=0x7f78ac16b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac15ac80 0x7f78ac16b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 -- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 msgr2=0x7f78ac167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 0x7f78ac167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 -- 172.21.15.98:0/3366878886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009480 con 0x7f78ac15a8b0 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b09c3700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac16b3a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f789c001eb0 tx=0x7f789c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00ce10 con 0x7f78ac15a8b0 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c011590 con 0x7f78ac15a8b0 2023-10-06T01:40:46.318 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c011950 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac0ac8f0 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.310+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac167a60 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.311+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac167c80 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.312+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f789c011d40 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.312+0000 7f78a17fa700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.312+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f789c0839a0 con 0x7f78ac15a8b0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.312+0000 7f78b11c4700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.319 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.313+0000 7f78b11c4700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 0x7f78900693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78a8007fd0 tx=0x7f78a8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.320 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.316+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f789c050970 con 0x7f78ac15a8b0 2023-10-06T01:40:46.426 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-10-06T01:40:46.425+0000 7f8dca44b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:40:46.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.496+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78ac0ad080 con 0x7f7890066f30 2023-10-06T01:40:46.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.505+0000 7f78a17fa700 1 -- 172.21.15.98:0/3366878886 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310213 (secure 0 0 0) 0x7f78ac0ad080 con 0x7f7890066f30 2023-10-06T01:40:46.511 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:46.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.508+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 msgr2=0x7f78900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.508+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 0x7f78900693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78a8007fd0 tx=0x7f78a8016000 comp rx=0 tx=0).stop 2023-10-06T01:40:46.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.508+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 msgr2=0x7f78ac16b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.508+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac16b3a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f789c001eb0 tx=0x7f789c007ec0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 shutdown_connections 2023-10-06T01:40:46.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7890066f30 0x7f78900693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac15ac80 0x7f78ac16b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac0c8f60 0x7f78ac167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 --2- 172.21.15.98:0/3366878886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac15a8b0 0x7f78ac16b3a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 >> 172.21.15.98:0/3366878886 conn(0x7f78ac067dd0 msgr2=0x7f78ac156130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 shutdown_connections 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.509+0000 7f78b3428700 1 -- 172.21.15.98:0/3366878886 wait complete. 2023-10-06T01:40:46.517 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:46.521 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T01:40:45.797527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382528,"kb_used_data":4224,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592192,"statfs":{"total":773094113280,"available":770654404608,"internally_reserved":0,"allocated":4325376,"data_stored":2535883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.349283"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478358+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:42.478358+0000","last_peered":"2023-10-06T01:40:42.478358+0000","last_clean":"2023-10-06T01:40:42.478358+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:42.478358+0000","last_undegraded":"2023-10-06T01:40:42.478358+0000","last_fullsized":"2023-10-06T01:40:42.478358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459724+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457109+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479027+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:42.479027+0000","last_peered":"2023-10-06T01:40:42.479027+0000","last_clean":"2023-10-06T01:40:42.479027+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:42.479027+0000","last_undegraded":"2023-10-06T01:40:42.479027+0000","last_fullsized":"2023-10-06T01:40:42.479027+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459074+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:06.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459386+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477100+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:42.477100+0000","last_peered":"2023-10-06T01:40:42.477100+0000","last_clean":"2023-10-06T01:40:42.477100+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:42.477100+0000","last_undegraded":"2023-10-06T01:40:42.477100+0000","last_fullsized":"2023-10-06T01:40:42.477100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479538+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:42.479538+0000","last_peered":"2023-10-06T01:40:42.479538+0000","last_clean":"2023-10-06T01:40:42.479538+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:42.479538+0000","last_undegraded":"2023-10-06T01:40:42.479538+0000","last_fullsized":"2023-10-06T01:40:42.479538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478626+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:42.478626+0000","last_peered":"2023-10-06T01:40:42.478626+0000","last_clean":"2023-10-06T01:40:42.478626+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:42.478626+0000","last_undegraded":"2023-10-06T01:40:42.478626+0000","last_fullsized":"2023-10-06T01:40:42.478626+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480200+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:42.480200+0000","last_peered":"2023-10-06T01:40:42.480200+0000","last_clean":"2023-10-06T01:40:42.480200+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:42.480200+0000","last_undegraded":"2023-10-06T01:40:42.480200+0000","last_fullsized":"2023-10-06T01:40:42.480200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481292+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:42.481292+0000","last_peered":"2023-10-06T01:40:42.481292+0000","last_clean":"2023-10-06T01:40:42.481292+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:42.481292+0000","last_undegraded":"2023-10-06T01:40:42.481292+0000","last_fullsized":"2023-10-06T01:40:42.481292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.460547+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457971+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:51.433736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480979+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:42.480979+0000","last_peered":"2023-10-06T01:40:42.480979+0000","last_clean":"2023-10-06T01:40:42.480979+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:42.480979+0000","last_undegraded":"2023-10-06T01:40:42.480979+0000","last_fullsized":"2023-10-06T01:40:42.480979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457286+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478135+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:42.478135+0000","last_peered":"2023-10-06T01:40:42.478135+0000","last_clean":"2023-10-06T01:40:42.478135+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:42.478135+0000","last_undegraded":"2023-10-06T01:40:42.478135+0000","last_fullsized":"2023-10-06T01:40:42.478135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455825+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456995+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482708+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:42.482708+0000","last_peered":"2023-10-06T01:40:42.482708+0000","last_clean":"2023-10-06T01:40:42.482708+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:42.482708+0000","last_undegraded":"2023-10-06T01:40:42.482708+0000","last_fullsized":"2023-10-06T01:40:42.482708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458747+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455840+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481147+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:42.481147+0000","last_peered":"2023-10-06T01:40:42.481147+0000","last_clean":"2023-10-06T01:40:42.481147+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:42.481147+0000","last_undegraded":"2023-10-06T01:40:42.481147+0000","last_fullsized":"2023-10-06T01:40:42.481147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.472954+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:42.472954+0000","last_peered":"2023-10-06T01:40:42.472954+0000","last_clean":"2023-10-06T01:40:42.472954+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:42.472954+0000","last_undegraded":"2023-10-06T01:40:42.472954+0000","last_fullsized":"2023-10-06T01:40:42.472954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473497+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:42.473497+0000","last_peered":"2023-10-06T01:40:42.473497+0000","last_clean":"2023-10-06T01:40:42.473497+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:42.473497+0000","last_undegraded":"2023-10-06T01:40:42.473497+0000","last_fullsized":"2023-10-06T01:40:42.473497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474680+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:42.474680+0000","last_peered":"2023-10-06T01:40:42.474680+0000","last_clean":"2023-10-06T01:40:42.474680+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:42.474680+0000","last_undegraded":"2023-10-06T01:40:42.474680+0000","last_fullsized":"2023-10-06T01:40:42.474680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458345+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456575+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.463947+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478517+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:42.478517+0000","last_peered":"2023-10-06T01:40:42.478517+0000","last_clean":"2023-10-06T01:40:42.478517+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:42.478517+0000","last_undegraded":"2023-10-06T01:40:42.478517+0000","last_fullsized":"2023-10-06T01:40:42.478517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455323+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455997+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457381+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:52.900640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456256+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467749+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:40:41.467749+0000","last_peered":"2023-10-06T01:40:41.467749+0000","last_clean":"2023-10-06T01:40:41.467749+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:40:41.467749+0000","last_undegraded":"2023-10-06T01:40:41.467749+0000","last_fullsized":"2023-10-06T01:40:41.467749+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465919+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:42.465919+0000","last_peered":"2023-10-06T01:40:42.465919+0000","last_clean":"2023-10-06T01:40:42.465919+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:42.465919+0000","last_undegraded":"2023-10-06T01:40:42.465919+0000","last_fullsized":"2023-10-06T01:40:42.465919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.472783+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:40:41.472783+0000","last_peered":"2023-10-06T01:40:41.472783+0000","last_clean":"2023-10-06T01:40:41.472783+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:40:41.472783+0000","last_undegraded":"2023-10-06T01:40:41.472783+0000","last_fullsized":"2023-10-06T01:40:41.472783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.461022+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470447+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:40:41.470447+0000","last_peered":"2023-10-06T01:40:41.470447+0000","last_clean":"2023-10-06T01:40:41.470447+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:40:41.470447+0000","last_undegraded":"2023-10-06T01:40:41.470447+0000","last_fullsized":"2023-10-06T01:40:41.470447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455453+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484424+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:40:42.484424+0000","last_peered":"2023-10-06T01:40:42.484424+0000","last_clean":"2023-10-06T01:40:42.484424+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:40:42.484424+0000","last_undegraded":"2023-10-06T01:40:42.484424+0000","last_fullsized":"2023-10-06T01:40:42.484424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454760+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464311+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:40:41.464311+0000","last_peered":"2023-10-06T01:40:41.464311+0000","last_clean":"2023-10-06T01:40:41.464311+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:40:41.464311+0000","last_undegraded":"2023-10-06T01:40:41.464311+0000","last_fullsized":"2023-10-06T01:40:41.464311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475400+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:42.475400+0000","last_peered":"2023-10-06T01:40:42.475400+0000","last_clean":"2023-10-06T01:40:42.475400+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:42.475400+0000","last_undegraded":"2023-10-06T01:40:42.475400+0000","last_fullsized":"2023-10-06T01:40:42.475400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468752+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:40:41.468752+0000","last_peered":"2023-10-06T01:40:41.468752+0000","last_clean":"2023-10-06T01:40:41.468752+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:40:41.468752+0000","last_undegraded":"2023-10-06T01:40:41.468752+0000","last_fullsized":"2023-10-06T01:40:41.468752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478912+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:42.478912+0000","last_peered":"2023-10-06T01:40:42.478912+0000","last_clean":"2023-10-06T01:40:42.478912+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:42.478912+0000","last_undegraded":"2023-10-06T01:40:42.478912+0000","last_fullsized":"2023-10-06T01:40:42.478912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467878+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:40:41.467878+0000","last_peered":"2023-10-06T01:40:41.467878+0000","last_clean":"2023-10-06T01:40:41.467878+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:40:41.467878+0000","last_undegraded":"2023-10-06T01:40:41.467878+0000","last_fullsized":"2023-10-06T01:40:41.467878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473067+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:42.473067+0000","last_peered":"2023-10-06T01:40:42.473067+0000","last_clean":"2023-10-06T01:40:42.473067+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:42.473067+0000","last_undegraded":"2023-10-06T01:40:42.473067+0000","last_fullsized":"2023-10-06T01:40:42.473067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470737+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:41.470737+0000","last_peered":"2023-10-06T01:40:41.470737+0000","last_clean":"2023-10-06T01:40:41.470737+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:41.470737+0000","last_undegraded":"2023-10-06T01:40:41.470737+0000","last_fullsized":"2023-10-06T01:40:41.470737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456224+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.463319+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:40:41.463319+0000","last_peered":"2023-10-06T01:40:41.463319+0000","last_clean":"2023-10-06T01:40:41.463319+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:40:41.463319+0000","last_undegraded":"2023-10-06T01:40:41.463319+0000","last_fullsized":"2023-10-06T01:40:41.463319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483351+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:42.483351+0000","last_peered":"2023-10-06T01:40:42.483351+0000","last_clean":"2023-10-06T01:40:42.483351+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:42.483351+0000","last_undegraded":"2023-10-06T01:40:42.483351+0000","last_fullsized":"2023-10-06T01:40:42.483351+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471793+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:41.471793+0000","last_peered":"2023-10-06T01:40:41.471793+0000","last_clean":"2023-10-06T01:40:41.471793+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:41.471793+0000","last_undegraded":"2023-10-06T01:40:41.471793+0000","last_fullsized":"2023-10-06T01:40:41.471793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455708+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490915+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:40:42.490915+0000","last_peered":"2023-10-06T01:40:42.490915+0000","last_clean":"2023-10-06T01:40:42.490915+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:40:42.490915+0000","last_undegraded":"2023-10-06T01:40:42.490915+0000","last_fullsized":"2023-10-06T01:40:42.490915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478379+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:42.478379+0000","last_peered":"2023-10-06T01:40:42.478379+0000","last_clean":"2023-10-06T01:40:42.478379+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:42.478379+0000","last_undegraded":"2023-10-06T01:40:42.478379+0000","last_fullsized":"2023-10-06T01:40:42.478379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465691+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:40:41.465691+0000","last_peered":"2023-10-06T01:40:41.465691+0000","last_clean":"2023-10-06T01:40:41.465691+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:40:41.465691+0000","last_undegraded":"2023-10-06T01:40:41.465691+0000","last_fullsized":"2023-10-06T01:40:41.465691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484065+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:42.484065+0000","last_peered":"2023-10-06T01:40:42.484065+0000","last_clean":"2023-10-06T01:40:42.484065+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:42.484065+0000","last_undegraded":"2023-10-06T01:40:42.484065+0000","last_fullsized":"2023-10-06T01:40:42.484065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468974+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:40:41.468974+0000","last_peered":"2023-10-06T01:40:41.468974+0000","last_clean":"2023-10-06T01:40:41.468974+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:40:41.468974+0000","last_undegraded":"2023-10-06T01:40:41.468974+0000","last_fullsized":"2023-10-06T01:40:41.468974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478175+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:42.478175+0000","last_peered":"2023-10-06T01:40:42.478175+0000","last_clean":"2023-10-06T01:40:42.478175+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:42.478175+0000","last_undegraded":"2023-10-06T01:40:42.478175+0000","last_fullsized":"2023-10-06T01:40:42.478175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469395+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:41.469395+0000","last_peered":"2023-10-06T01:40:41.469395+0000","last_clean":"2023-10-06T01:40:41.469395+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:41.469395+0000","last_undegraded":"2023-10-06T01:40:41.469395+0000","last_fullsized":"2023-10-06T01:40:41.469395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455404+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467708+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:40:41.467708+0000","last_peered":"2023-10-06T01:40:41.467708+0000","last_clean":"2023-10-06T01:40:41.467708+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:40:41.467708+0000","last_undegraded":"2023-10-06T01:40:41.467708+0000","last_fullsized":"2023-10-06T01:40:41.467708+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481908+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:42.481908+0000","last_peered":"2023-10-06T01:40:42.481908+0000","last_clean":"2023-10-06T01:40:42.481908+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:42.481908+0000","last_undegraded":"2023-10-06T01:40:42.481908+0000","last_fullsized":"2023-10-06T01:40:42.481908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491752+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:40:42.491752+0000","last_peered":"2023-10-06T01:40:42.491752+0000","last_clean":"2023-10-06T01:40:42.491752+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:40:42.491752+0000","last_undegraded":"2023-10-06T01:40:42.491752+0000","last_fullsized":"2023-10-06T01:40:42.491752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457872+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:03.458529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.466682+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:40:41.466682+0000","last_peered":"2023-10-06T01:40:41.466682+0000","last_clean":"2023-10-06T01:40:41.466682+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:40:41.466682+0000","last_undegraded":"2023-10-06T01:40:41.466682+0000","last_fullsized":"2023-10-06T01:40:41.466682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483508+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:42.483508+0000","last_peered":"2023-10-06T01:40:42.483508+0000","last_clean":"2023-10-06T01:40:42.483508+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:42.483508+0000","last_undegraded":"2023-10-06T01:40:42.483508+0000","last_fullsized":"2023-10-06T01:40:42.483508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465545+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:40:41.465545+0000","last_peered":"2023-10-06T01:40:41.465545+0000","last_clean":"2023-10-06T01:40:41.465545+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:40:41.465545+0000","last_undegraded":"2023-10-06T01:40:41.465545+0000","last_fullsized":"2023-10-06T01:40:41.465545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474713+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:42.474713+0000","last_peered":"2023-10-06T01:40:42.474713+0000","last_clean":"2023-10-06T01:40:42.474713+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:42.474713+0000","last_undegraded":"2023-10-06T01:40:42.474713+0000","last_fullsized":"2023-10-06T01:40:42.474713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468411+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:40:41.468411+0000","last_peered":"2023-10-06T01:40:41.468411+0000","last_clean":"2023-10-06T01:40:41.468411+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:40:41.468411+0000","last_undegraded":"2023-10-06T01:40:41.468411+0000","last_fullsized":"2023-10-06T01:40:41.468411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.471507+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:42.471507+0000","last_peered":"2023-10-06T01:40:42.471507+0000","last_clean":"2023-10-06T01:40:42.471507+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:42.471507+0000","last_undegraded":"2023-10-06T01:40:42.471507+0000","last_fullsized":"2023-10-06T01:40:42.471507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468147+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:40:41.468147+0000","last_peered":"2023-10-06T01:40:41.468147+0000","last_clean":"2023-10-06T01:40:41.468147+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:40:41.468147+0000","last_undegraded":"2023-10-06T01:40:41.468147+0000","last_fullsized":"2023-10-06T01:40:41.468147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456078+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464033+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:40:41.464033+0000","last_peered":"2023-10-06T01:40:41.464033+0000","last_clean":"2023-10-06T01:40:41.464033+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:40:41.464033+0000","last_undegraded":"2023-10-06T01:40:41.464033+0000","last_fullsized":"2023-10-06T01:40:41.464033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455168+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.473889+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:40:41.473889+0000","last_peered":"2023-10-06T01:40:41.473889+0000","last_clean":"2023-10-06T01:40:41.473889+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:40:41.473889+0000","last_undegraded":"2023-10-06T01:40:41.473889+0000","last_fullsized":"2023-10-06T01:40:41.473889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456925+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:23.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.474303+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:41.474303+0000","last_peered":"2023-10-06T01:40:41.474303+0000","last_clean":"2023-10-06T01:40:41.474303+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:41.474303+0000","last_undegraded":"2023-10-06T01:40:41.474303+0000","last_fullsized":"2023-10-06T01:40:41.474303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478522+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:42.478522+0000","last_peered":"2023-10-06T01:40:42.478522+0000","last_clean":"2023-10-06T01:40:42.478522+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:42.478522+0000","last_undegraded":"2023-10-06T01:40:42.478522+0000","last_fullsized":"2023-10-06T01:40:42.478522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469403+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:41.469403+0000","last_peered":"2023-10-06T01:40:41.469403+0000","last_clean":"2023-10-06T01:40:41.469403+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:41.469403+0000","last_undegraded":"2023-10-06T01:40:41.469403+0000","last_fullsized":"2023-10-06T01:40:41.469403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454931+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464545+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:40:41.464545+0000","last_peered":"2023-10-06T01:40:41.464545+0000","last_clean":"2023-10-06T01:40:41.464545+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:40:41.464545+0000","last_undegraded":"2023-10-06T01:40:41.464545+0000","last_fullsized":"2023-10-06T01:40:41.464545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456794+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:40:40.456794+0000","last_peered":"2023-10-06T01:40:40.456794+0000","last_clean":"2023-10-06T01:40:40.456794+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:40:40.456794+0000","last_undegraded":"2023-10-06T01:40:40.456794+0000","last_fullsized":"2023-10-06T01:40:40.456794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454856+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477701+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:41.477701+0000","last_peered":"2023-10-06T01:40:41.477701+0000","last_clean":"2023-10-06T01:40:41.477701+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:41.477701+0000","last_undegraded":"2023-10-06T01:40:41.477701+0000","last_fullsized":"2023-10-06T01:40:41.477701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454750+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:40:40.454750+0000","last_peered":"2023-10-06T01:40:40.454750+0000","last_clean":"2023-10-06T01:40:40.454750+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:40:40.454750+0000","last_undegraded":"2023-10-06T01:40:40.454750+0000","last_fullsized":"2023-10-06T01:40:40.454750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475832+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:42.475832+0000","last_peered":"2023-10-06T01:40:42.475832+0000","last_clean":"2023-10-06T01:40:42.475832+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:42.475832+0000","last_undegraded":"2023-10-06T01:40:42.475832+0000","last_fullsized":"2023-10-06T01:40:42.475832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470594+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:40:41.470594+0000","last_peered":"2023-10-06T01:40:41.470594+0000","last_clean":"2023-10-06T01:40:41.470594+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:40:41.470594+0000","last_undegraded":"2023-10-06T01:40:41.470594+0000","last_fullsized":"2023-10-06T01:40:41.470594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454576+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:40:40.454576+0000","last_peered":"2023-10-06T01:40:40.454576+0000","last_clean":"2023-10-06T01:40:40.454576+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:40:40.454576+0000","last_undegraded":"2023-10-06T01:40:40.454576+0000","last_fullsized":"2023-10-06T01:40:40.454576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456861+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:41.456861+0000","last_peered":"2023-10-06T01:40:41.456861+0000","last_clean":"2023-10-06T01:40:41.456861+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:41.456861+0000","last_undegraded":"2023-10-06T01:40:41.456861+0000","last_fullsized":"2023-10-06T01:40:41.456861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479190+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:42.479190+0000","last_peered":"2023-10-06T01:40:42.479190+0000","last_clean":"2023-10-06T01:40:42.479190+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:42.479190+0000","last_undegraded":"2023-10-06T01:40:42.479190+0000","last_fullsized":"2023-10-06T01:40:42.479190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477681+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:41.477681+0000","last_peered":"2023-10-06T01:40:41.477681+0000","last_clean":"2023-10-06T01:40:41.477681+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:41.477681+0000","last_undegraded":"2023-10-06T01:40:41.477681+0000","last_fullsized":"2023-10-06T01:40:41.477681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.458496+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:41.458496+0000","last_peered":"2023-10-06T01:40:41.458496+0000","last_clean":"2023-10-06T01:40:41.458496+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:41.458496+0000","last_undegraded":"2023-10-06T01:40:41.458496+0000","last_fullsized":"2023-10-06T01:40:41.458496+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480021+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:42.480021+0000","last_peered":"2023-10-06T01:40:42.480021+0000","last_clean":"2023-10-06T01:40:42.480021+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:42.480021+0000","last_undegraded":"2023-10-06T01:40:42.480021+0000","last_fullsized":"2023-10-06T01:40:42.480021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468486+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:40:41.468486+0000","last_peered":"2023-10-06T01:40:41.468486+0000","last_clean":"2023-10-06T01:40:41.468486+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:40:41.468486+0000","last_undegraded":"2023-10-06T01:40:41.468486+0000","last_fullsized":"2023-10-06T01:40:41.468486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456774+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:41.456774+0000","last_peered":"2023-10-06T01:40:41.456774+0000","last_clean":"2023-10-06T01:40:41.456774+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:41.456774+0000","last_undegraded":"2023-10-06T01:40:41.456774+0000","last_fullsized":"2023-10-06T01:40:41.456774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485111+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:42.485111+0000","last_peered":"2023-10-06T01:40:42.485111+0000","last_clean":"2023-10-06T01:40:42.485111+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:42.485111+0000","last_undegraded":"2023-10-06T01:40:42.485111+0000","last_fullsized":"2023-10-06T01:40:42.485111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.462919+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:40:41.462919+0000","last_peered":"2023-10-06T01:40:41.462919+0000","last_clean":"2023-10-06T01:40:41.462919+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:40:41.462919+0000","last_undegraded":"2023-10-06T01:40:41.462919+0000","last_fullsized":"2023-10-06T01:40:41.462919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456248+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:40:40.456248+0000","last_peered":"2023-10-06T01:40:40.456248+0000","last_clean":"2023-10-06T01:40:40.456248+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:40:40.456248+0000","last_undegraded":"2023-10-06T01:40:40.456248+0000","last_fullsized":"2023-10-06T01:40:40.456248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459939+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.461788+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:40:41.461788+0000","last_peered":"2023-10-06T01:40:41.461788+0000","last_clean":"2023-10-06T01:40:41.461788+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:40:41.461788+0000","last_undegraded":"2023-10-06T01:40:41.461788+0000","last_fullsized":"2023-10-06T01:40:41.461788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469064+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:41.469064+0000","last_peered":"2023-10-06T01:40:41.469064+0000","last_clean":"2023-10-06T01:40:41.469064+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:41.469064+0000","last_undegraded":"2023-10-06T01:40:41.469064+0000","last_fullsized":"2023-10-06T01:40:41.469064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455484+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471786+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:41.471786+0000","last_peered":"2023-10-06T01:40:41.471786+0000","last_clean":"2023-10-06T01:40:41.471786+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:41.471786+0000","last_undegraded":"2023-10-06T01:40:41.471786+0000","last_fullsized":"2023-10-06T01:40:41.471786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469790+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:41.469790+0000","last_peered":"2023-10-06T01:40:41.469790+0000","last_clean":"2023-10-06T01:40:41.469790+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:41.469790+0000","last_undegraded":"2023-10-06T01:40:41.469790+0000","last_fullsized":"2023-10-06T01:40:41.469790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":766304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297688,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074152,"statfs":{"total":96636764160,"available":96331931648,"internally_reserved":0,"allocated":409600,"data_stored":176490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":751850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.254},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":175880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":167057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.249},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.307},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.378}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.622}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.309},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.624}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":161482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.372},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34100000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:46.541 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:46.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.674+0000 7f3960eca700 1 Processor -- start 2023-10-06T01:40:46.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.674+0000 7f3960eca700 1 -- start start 2023-10-06T01:40:46.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3960eca700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f395c0b07a0 0x7f395c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3960eca700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395c0b0ca0 con 0x7f395c0b49e0 2023-10-06T01:40:46.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3960eca700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395c0b0f80 con 0x7f395c1595a0 2023-10-06T01:40:46.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3960eca700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395c0ab390 con 0x7f395c0b07a0 2023-10-06T01:40:46.677 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f395ad9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f395c0b07a0 0x7f395c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3959d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f395c0b49e0 0x7f395c0aac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45646/0 (socket says 172.21.15.98:45646) 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.675+0000 7f3959d9b700 1 -- 172.21.15.98:0/1421896731 learned_addr learned my addr 172.21.15.98:0/1421896731 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395ad9d700 1 -- 172.21.15.98:0/1421896731 >> v1:172.21.15.184:6789/0 conn(0x7f395c1595a0 legacy=0x7f395c0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395ad9d700 1 -- 172.21.15.98:0/1421896731 >> v1:172.21.15.98:6789/0 conn(0x7f395c0b49e0 legacy=0x7f395c0aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395ad9d700 1 -- 172.21.15.98:0/1421896731 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0b1260 con 0x7f395c0b07a0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395959a700 1 -- 172.21.15.98:0/1421896731 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449296851 0 0) 0x7f395c0b0ca0 con 0x7f395c0b49e0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395ad9d700 1 --2- 172.21.15.98:0/1421896731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c0b07a0 0x7f395c0ccfd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f394c009e90 tx=0x7f394c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f31000e7733ce1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395959a700 1 -- 172.21.15.98:0/1421896731 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00e070 con 0x7f395c0b07a0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395959a700 1 -- 172.21.15.98:0/1421896731 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00bbb0 con 0x7f395c0b07a0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f395959a700 1 -- 172.21.15.98:0/1421896731 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c012430 con 0x7f395c0b07a0 2023-10-06T01:40:46.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f3960eca700 1 -- 172.21.15.98:0/1421896731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c0b07a0 msgr2=0x7f395c0ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.676+0000 7f3960eca700 1 --2- 172.21.15.98:0/1421896731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c0b07a0 0x7f395c0ccfd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f394c009e90 tx=0x7f394c009780 comp rx=0 tx=0).stop 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.677+0000 7f3960eca700 1 -- 172.21.15.98:0/1421896731 shutdown_connections 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.677+0000 7f3960eca700 1 --2- 172.21.15.98:0/1421896731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c0b07a0 0x7f395c0ccfd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.677+0000 7f3960eca700 1 -- 172.21.15.98:0/1421896731 >> 172.21.15.98:0/1421896731 conn(0x7f395c099830 msgr2=0x7f395c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.677+0000 7f3960eca700 1 -- 172.21.15.98:0/1421896731 shutdown_connections 2023-10-06T01:40:46.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.677+0000 7f3960eca700 1 -- 172.21.15.98:0/1421896731 wait complete. 2023-10-06T01:40:46.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 Processor -- start 2023-10-06T01:40:46.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 -- start start 2023-10-06T01:40:46.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f395c0b07a0 0x7f395c15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f395c0b49e0 0x7f395c157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c158050 con 0x7f395c1595a0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c09aa40 con 0x7f395c0b49e0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f3960eca700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c09dbb0 con 0x7f395c0b07a0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f395ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f395ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46988/0 (socket says 172.21.15.98:46988) 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.678+0000 7f395ad9d700 1 -- 172.21.15.98:0/3399736923 learned_addr learned my addr 172.21.15.98:0/3399736923 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395a59c700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f395c0b07a0 0x7f395c15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f3959d9b700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f395c0b49e0 0x7f395c157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 -- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f395c0b07a0 msgr2=0x7f395c15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f395c0b07a0 0x7f395c15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 -- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f395c0b49e0 msgr2=0x7f395c157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f395c0b49e0 0x7f395c157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 -- 172.21.15.98:0/3399736923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c009a20 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f395ad9d700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f395c067c60 tx=0x7f394c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c01e710 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c01e8b0 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c09acb0 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c01ec70 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.679+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c09b040 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.680+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c064ad0 con 0x7f395c1595a0 2023-10-06T01:40:46.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.680+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f394c024070 con 0x7f395c1595a0 2023-10-06T01:40:46.684 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.681+0000 7f39527fc700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 0x7f393c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:46.684 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.681+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f394c087450 con 0x7f395c1595a0 2023-10-06T01:40:46.684 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.681+0000 7f395a59c700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 0x7f393c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:46.684 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.681+0000 7f395a59c700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 0x7f393c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3954009930 tx=0x7f3954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:46.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.685+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f394c054420 con 0x7f395c1595a0 2023-10-06T01:40:46.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.862+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f395c09b1d0 con 0x7f393c066f30 2023-10-06T01:40:46.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.871+0000 7f39527fc700 1 -- 172.21.15.98:0/3399736923 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310213 (secure 0 0 0) 0x7f395c09b1d0 con 0x7f393c066f30 2023-10-06T01:40:46.875 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 msgr2=0x7f393c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 0x7f393c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3954009930 tx=0x7f3954008040 comp rx=0 tx=0).stop 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 msgr2=0x7f395c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f395c067c60 tx=0x7f394c018000 comp rx=0 tx=0).stop 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 shutdown_connections 2023-10-06T01:40:46.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f393c066f30 0x7f393c0693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395c1595a0 0x7f395c15c100 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f395c0b07a0 0x7f395c15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 --2- 172.21.15.98:0/3399736923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f395c0b49e0 0x7f395c157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 >> 172.21.15.98:0/3399736923 conn(0x7f395c099830 msgr2=0x7f395c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 shutdown_connections 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:46.874+0000 7f3960eca700 1 -- 172.21.15.98:0/3399736923 wait complete. 2023-10-06T01:40:46.878 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:46.886 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T01:40:45.797527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382528,"kb_used_data":4224,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592192,"statfs":{"total":773094113280,"available":770654404608,"internally_reserved":0,"allocated":4325376,"data_stored":2535883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.349283"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478358+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:42.478358+0000","last_peered":"2023-10-06T01:40:42.478358+0000","last_clean":"2023-10-06T01:40:42.478358+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:42.478358+0000","last_undegraded":"2023-10-06T01:40:42.478358+0000","last_fullsized":"2023-10-06T01:40:42.478358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459724+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457109+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479027+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:42.479027+0000","last_peered":"2023-10-06T01:40:42.479027+0000","last_clean":"2023-10-06T01:40:42.479027+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:42.479027+0000","last_undegraded":"2023-10-06T01:40:42.479027+0000","last_fullsized":"2023-10-06T01:40:42.479027+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459074+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:06.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459386+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477100+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:42.477100+0000","last_peered":"2023-10-06T01:40:42.477100+0000","last_clean":"2023-10-06T01:40:42.477100+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:42.477100+0000","last_undegraded":"2023-10-06T01:40:42.477100+0000","last_fullsized":"2023-10-06T01:40:42.477100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479538+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:42.479538+0000","last_peered":"2023-10-06T01:40:42.479538+0000","last_clean":"2023-10-06T01:40:42.479538+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:42.479538+0000","last_undegraded":"2023-10-06T01:40:42.479538+0000","last_fullsized":"2023-10-06T01:40:42.479538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478626+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:42.478626+0000","last_peered":"2023-10-06T01:40:42.478626+0000","last_clean":"2023-10-06T01:40:42.478626+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:42.478626+0000","last_undegraded":"2023-10-06T01:40:42.478626+0000","last_fullsized":"2023-10-06T01:40:42.478626+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480200+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:42.480200+0000","last_peered":"2023-10-06T01:40:42.480200+0000","last_clean":"2023-10-06T01:40:42.480200+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:42.480200+0000","last_undegraded":"2023-10-06T01:40:42.480200+0000","last_fullsized":"2023-10-06T01:40:42.480200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481292+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:42.481292+0000","last_peered":"2023-10-06T01:40:42.481292+0000","last_clean":"2023-10-06T01:40:42.481292+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:42.481292+0000","last_undegraded":"2023-10-06T01:40:42.481292+0000","last_fullsized":"2023-10-06T01:40:42.481292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.460547+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457971+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:51.433736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480979+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:42.480979+0000","last_peered":"2023-10-06T01:40:42.480979+0000","last_clean":"2023-10-06T01:40:42.480979+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:42.480979+0000","last_undegraded":"2023-10-06T01:40:42.480979+0000","last_fullsized":"2023-10-06T01:40:42.480979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457286+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478135+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:42.478135+0000","last_peered":"2023-10-06T01:40:42.478135+0000","last_clean":"2023-10-06T01:40:42.478135+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:42.478135+0000","last_undegraded":"2023-10-06T01:40:42.478135+0000","last_fullsized":"2023-10-06T01:40:42.478135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455825+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456995+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482708+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:42.482708+0000","last_peered":"2023-10-06T01:40:42.482708+0000","last_clean":"2023-10-06T01:40:42.482708+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:42.482708+0000","last_undegraded":"2023-10-06T01:40:42.482708+0000","last_fullsized":"2023-10-06T01:40:42.482708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458747+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455840+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481147+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:42.481147+0000","last_peered":"2023-10-06T01:40:42.481147+0000","last_clean":"2023-10-06T01:40:42.481147+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:42.481147+0000","last_undegraded":"2023-10-06T01:40:42.481147+0000","last_fullsized":"2023-10-06T01:40:42.481147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.472954+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:42.472954+0000","last_peered":"2023-10-06T01:40:42.472954+0000","last_clean":"2023-10-06T01:40:42.472954+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:42.472954+0000","last_undegraded":"2023-10-06T01:40:42.472954+0000","last_fullsized":"2023-10-06T01:40:42.472954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473497+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:42.473497+0000","last_peered":"2023-10-06T01:40:42.473497+0000","last_clean":"2023-10-06T01:40:42.473497+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:42.473497+0000","last_undegraded":"2023-10-06T01:40:42.473497+0000","last_fullsized":"2023-10-06T01:40:42.473497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474680+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:42.474680+0000","last_peered":"2023-10-06T01:40:42.474680+0000","last_clean":"2023-10-06T01:40:42.474680+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:42.474680+0000","last_undegraded":"2023-10-06T01:40:42.474680+0000","last_fullsized":"2023-10-06T01:40:42.474680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458345+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456575+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.463947+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478517+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:42.478517+0000","last_peered":"2023-10-06T01:40:42.478517+0000","last_clean":"2023-10-06T01:40:42.478517+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:42.478517+0000","last_undegraded":"2023-10-06T01:40:42.478517+0000","last_fullsized":"2023-10-06T01:40:42.478517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455323+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455997+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457381+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:52.900640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456256+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467749+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:40:41.467749+0000","last_peered":"2023-10-06T01:40:41.467749+0000","last_clean":"2023-10-06T01:40:41.467749+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:40:41.467749+0000","last_undegraded":"2023-10-06T01:40:41.467749+0000","last_fullsized":"2023-10-06T01:40:41.467749+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465919+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:42.465919+0000","last_peered":"2023-10-06T01:40:42.465919+0000","last_clean":"2023-10-06T01:40:42.465919+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:42.465919+0000","last_undegraded":"2023-10-06T01:40:42.465919+0000","last_fullsized":"2023-10-06T01:40:42.465919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.472783+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:40:41.472783+0000","last_peered":"2023-10-06T01:40:41.472783+0000","last_clean":"2023-10-06T01:40:41.472783+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:40:41.472783+0000","last_undegraded":"2023-10-06T01:40:41.472783+0000","last_fullsized":"2023-10-06T01:40:41.472783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.461022+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470447+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:40:41.470447+0000","last_peered":"2023-10-06T01:40:41.470447+0000","last_clean":"2023-10-06T01:40:41.470447+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:40:41.470447+0000","last_undegraded":"2023-10-06T01:40:41.470447+0000","last_fullsized":"2023-10-06T01:40:41.470447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455453+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484424+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:40:42.484424+0000","last_peered":"2023-10-06T01:40:42.484424+0000","last_clean":"2023-10-06T01:40:42.484424+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:40:42.484424+0000","last_undegraded":"2023-10-06T01:40:42.484424+0000","last_fullsized":"2023-10-06T01:40:42.484424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454760+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464311+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:40:41.464311+0000","last_peered":"2023-10-06T01:40:41.464311+0000","last_clean":"2023-10-06T01:40:41.464311+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:40:41.464311+0000","last_undegraded":"2023-10-06T01:40:41.464311+0000","last_fullsized":"2023-10-06T01:40:41.464311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475400+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:42.475400+0000","last_peered":"2023-10-06T01:40:42.475400+0000","last_clean":"2023-10-06T01:40:42.475400+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:42.475400+0000","last_undegraded":"2023-10-06T01:40:42.475400+0000","last_fullsized":"2023-10-06T01:40:42.475400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468752+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:40:41.468752+0000","last_peered":"2023-10-06T01:40:41.468752+0000","last_clean":"2023-10-06T01:40:41.468752+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:40:41.468752+0000","last_undegraded":"2023-10-06T01:40:41.468752+0000","last_fullsized":"2023-10-06T01:40:41.468752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478912+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:42.478912+0000","last_peered":"2023-10-06T01:40:42.478912+0000","last_clean":"2023-10-06T01:40:42.478912+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:42.478912+0000","last_undegraded":"2023-10-06T01:40:42.478912+0000","last_fullsized":"2023-10-06T01:40:42.478912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467878+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:40:41.467878+0000","last_peered":"2023-10-06T01:40:41.467878+0000","last_clean":"2023-10-06T01:40:41.467878+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:40:41.467878+0000","last_undegraded":"2023-10-06T01:40:41.467878+0000","last_fullsized":"2023-10-06T01:40:41.467878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473067+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:42.473067+0000","last_peered":"2023-10-06T01:40:42.473067+0000","last_clean":"2023-10-06T01:40:42.473067+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:42.473067+0000","last_undegraded":"2023-10-06T01:40:42.473067+0000","last_fullsized":"2023-10-06T01:40:42.473067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470737+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:41.470737+0000","last_peered":"2023-10-06T01:40:41.470737+0000","last_clean":"2023-10-06T01:40:41.470737+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:41.470737+0000","last_undegraded":"2023-10-06T01:40:41.470737+0000","last_fullsized":"2023-10-06T01:40:41.470737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456224+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.463319+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:40:41.463319+0000","last_peered":"2023-10-06T01:40:41.463319+0000","last_clean":"2023-10-06T01:40:41.463319+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:40:41.463319+0000","last_undegraded":"2023-10-06T01:40:41.463319+0000","last_fullsized":"2023-10-06T01:40:41.463319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483351+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:42.483351+0000","last_peered":"2023-10-06T01:40:42.483351+0000","last_clean":"2023-10-06T01:40:42.483351+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:42.483351+0000","last_undegraded":"2023-10-06T01:40:42.483351+0000","last_fullsized":"2023-10-06T01:40:42.483351+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471793+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:41.471793+0000","last_peered":"2023-10-06T01:40:41.471793+0000","last_clean":"2023-10-06T01:40:41.471793+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:41.471793+0000","last_undegraded":"2023-10-06T01:40:41.471793+0000","last_fullsized":"2023-10-06T01:40:41.471793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455708+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490915+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:40:42.490915+0000","last_peered":"2023-10-06T01:40:42.490915+0000","last_clean":"2023-10-06T01:40:42.490915+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:40:42.490915+0000","last_undegraded":"2023-10-06T01:40:42.490915+0000","last_fullsized":"2023-10-06T01:40:42.490915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478379+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:42.478379+0000","last_peered":"2023-10-06T01:40:42.478379+0000","last_clean":"2023-10-06T01:40:42.478379+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:42.478379+0000","last_undegraded":"2023-10-06T01:40:42.478379+0000","last_fullsized":"2023-10-06T01:40:42.478379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465691+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:40:41.465691+0000","last_peered":"2023-10-06T01:40:41.465691+0000","last_clean":"2023-10-06T01:40:41.465691+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:40:41.465691+0000","last_undegraded":"2023-10-06T01:40:41.465691+0000","last_fullsized":"2023-10-06T01:40:41.465691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484065+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:42.484065+0000","last_peered":"2023-10-06T01:40:42.484065+0000","last_clean":"2023-10-06T01:40:42.484065+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:42.484065+0000","last_undegraded":"2023-10-06T01:40:42.484065+0000","last_fullsized":"2023-10-06T01:40:42.484065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468974+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:40:41.468974+0000","last_peered":"2023-10-06T01:40:41.468974+0000","last_clean":"2023-10-06T01:40:41.468974+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:40:41.468974+0000","last_undegraded":"2023-10-06T01:40:41.468974+0000","last_fullsized":"2023-10-06T01:40:41.468974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478175+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:42.478175+0000","last_peered":"2023-10-06T01:40:42.478175+0000","last_clean":"2023-10-06T01:40:42.478175+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:42.478175+0000","last_undegraded":"2023-10-06T01:40:42.478175+0000","last_fullsized":"2023-10-06T01:40:42.478175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469395+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:41.469395+0000","last_peered":"2023-10-06T01:40:41.469395+0000","last_clean":"2023-10-06T01:40:41.469395+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:41.469395+0000","last_undegraded":"2023-10-06T01:40:41.469395+0000","last_fullsized":"2023-10-06T01:40:41.469395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455404+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467708+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:40:41.467708+0000","last_peered":"2023-10-06T01:40:41.467708+0000","last_clean":"2023-10-06T01:40:41.467708+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:40:41.467708+0000","last_undegraded":"2023-10-06T01:40:41.467708+0000","last_fullsized":"2023-10-06T01:40:41.467708+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481908+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:42.481908+0000","last_peered":"2023-10-06T01:40:42.481908+0000","last_clean":"2023-10-06T01:40:42.481908+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:42.481908+0000","last_undegraded":"2023-10-06T01:40:42.481908+0000","last_fullsized":"2023-10-06T01:40:42.481908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491752+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:40:42.491752+0000","last_peered":"2023-10-06T01:40:42.491752+0000","last_clean":"2023-10-06T01:40:42.491752+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:40:42.491752+0000","last_undegraded":"2023-10-06T01:40:42.491752+0000","last_fullsized":"2023-10-06T01:40:42.491752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457872+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:03.458529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.466682+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:40:41.466682+0000","last_peered":"2023-10-06T01:40:41.466682+0000","last_clean":"2023-10-06T01:40:41.466682+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:40:41.466682+0000","last_undegraded":"2023-10-06T01:40:41.466682+0000","last_fullsized":"2023-10-06T01:40:41.466682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483508+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:42.483508+0000","last_peered":"2023-10-06T01:40:42.483508+0000","last_clean":"2023-10-06T01:40:42.483508+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:42.483508+0000","last_undegraded":"2023-10-06T01:40:42.483508+0000","last_fullsized":"2023-10-06T01:40:42.483508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465545+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:40:41.465545+0000","last_peered":"2023-10-06T01:40:41.465545+0000","last_clean":"2023-10-06T01:40:41.465545+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:40:41.465545+0000","last_undegraded":"2023-10-06T01:40:41.465545+0000","last_fullsized":"2023-10-06T01:40:41.465545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474713+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:42.474713+0000","last_peered":"2023-10-06T01:40:42.474713+0000","last_clean":"2023-10-06T01:40:42.474713+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:42.474713+0000","last_undegraded":"2023-10-06T01:40:42.474713+0000","last_fullsized":"2023-10-06T01:40:42.474713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468411+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:40:41.468411+0000","last_peered":"2023-10-06T01:40:41.468411+0000","last_clean":"2023-10-06T01:40:41.468411+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:40:41.468411+0000","last_undegraded":"2023-10-06T01:40:41.468411+0000","last_fullsized":"2023-10-06T01:40:41.468411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.471507+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:42.471507+0000","last_peered":"2023-10-06T01:40:42.471507+0000","last_clean":"2023-10-06T01:40:42.471507+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:42.471507+0000","last_undegraded":"2023-10-06T01:40:42.471507+0000","last_fullsized":"2023-10-06T01:40:42.471507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468147+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:40:41.468147+0000","last_peered":"2023-10-06T01:40:41.468147+0000","last_clean":"2023-10-06T01:40:41.468147+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:40:41.468147+0000","last_undegraded":"2023-10-06T01:40:41.468147+0000","last_fullsized":"2023-10-06T01:40:41.468147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456078+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464033+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:40:41.464033+0000","last_peered":"2023-10-06T01:40:41.464033+0000","last_clean":"2023-10-06T01:40:41.464033+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:40:41.464033+0000","last_undegraded":"2023-10-06T01:40:41.464033+0000","last_fullsized":"2023-10-06T01:40:41.464033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455168+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.473889+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:40:41.473889+0000","last_peered":"2023-10-06T01:40:41.473889+0000","last_clean":"2023-10-06T01:40:41.473889+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:40:41.473889+0000","last_undegraded":"2023-10-06T01:40:41.473889+0000","last_fullsized":"2023-10-06T01:40:41.473889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456925+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:23.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.474303+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:41.474303+0000","last_peered":"2023-10-06T01:40:41.474303+0000","last_clean":"2023-10-06T01:40:41.474303+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:41.474303+0000","last_undegraded":"2023-10-06T01:40:41.474303+0000","last_fullsized":"2023-10-06T01:40:41.474303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478522+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:42.478522+0000","last_peered":"2023-10-06T01:40:42.478522+0000","last_clean":"2023-10-06T01:40:42.478522+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:42.478522+0000","last_undegraded":"2023-10-06T01:40:42.478522+0000","last_fullsized":"2023-10-06T01:40:42.478522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469403+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:41.469403+0000","last_peered":"2023-10-06T01:40:41.469403+0000","last_clean":"2023-10-06T01:40:41.469403+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:41.469403+0000","last_undegraded":"2023-10-06T01:40:41.469403+0000","last_fullsized":"2023-10-06T01:40:41.469403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454931+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464545+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:40:41.464545+0000","last_peered":"2023-10-06T01:40:41.464545+0000","last_clean":"2023-10-06T01:40:41.464545+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:40:41.464545+0000","last_undegraded":"2023-10-06T01:40:41.464545+0000","last_fullsized":"2023-10-06T01:40:41.464545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456794+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:40:40.456794+0000","last_peered":"2023-10-06T01:40:40.456794+0000","last_clean":"2023-10-06T01:40:40.456794+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:40:40.456794+0000","last_undegraded":"2023-10-06T01:40:40.456794+0000","last_fullsized":"2023-10-06T01:40:40.456794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454856+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477701+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:41.477701+0000","last_peered":"2023-10-06T01:40:41.477701+0000","last_clean":"2023-10-06T01:40:41.477701+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:41.477701+0000","last_undegraded":"2023-10-06T01:40:41.477701+0000","last_fullsized":"2023-10-06T01:40:41.477701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454750+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:40:40.454750+0000","last_peered":"2023-10-06T01:40:40.454750+0000","last_clean":"2023-10-06T01:40:40.454750+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:40:40.454750+0000","last_undegraded":"2023-10-06T01:40:40.454750+0000","last_fullsized":"2023-10-06T01:40:40.454750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475832+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:42.475832+0000","last_peered":"2023-10-06T01:40:42.475832+0000","last_clean":"2023-10-06T01:40:42.475832+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:42.475832+0000","last_undegraded":"2023-10-06T01:40:42.475832+0000","last_fullsized":"2023-10-06T01:40:42.475832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470594+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:40:41.470594+0000","last_peered":"2023-10-06T01:40:41.470594+0000","last_clean":"2023-10-06T01:40:41.470594+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:40:41.470594+0000","last_undegraded":"2023-10-06T01:40:41.470594+0000","last_fullsized":"2023-10-06T01:40:41.470594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454576+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:40:40.454576+0000","last_peered":"2023-10-06T01:40:40.454576+0000","last_clean":"2023-10-06T01:40:40.454576+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:40:40.454576+0000","last_undegraded":"2023-10-06T01:40:40.454576+0000","last_fullsized":"2023-10-06T01:40:40.454576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456861+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:41.456861+0000","last_peered":"2023-10-06T01:40:41.456861+0000","last_clean":"2023-10-06T01:40:41.456861+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:41.456861+0000","last_undegraded":"2023-10-06T01:40:41.456861+0000","last_fullsized":"2023-10-06T01:40:41.456861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479190+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:42.479190+0000","last_peered":"2023-10-06T01:40:42.479190+0000","last_clean":"2023-10-06T01:40:42.479190+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:42.479190+0000","last_undegraded":"2023-10-06T01:40:42.479190+0000","last_fullsized":"2023-10-06T01:40:42.479190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477681+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:41.477681+0000","last_peered":"2023-10-06T01:40:41.477681+0000","last_clean":"2023-10-06T01:40:41.477681+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:41.477681+0000","last_undegraded":"2023-10-06T01:40:41.477681+0000","last_fullsized":"2023-10-06T01:40:41.477681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.458496+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:41.458496+0000","last_peered":"2023-10-06T01:40:41.458496+0000","last_clean":"2023-10-06T01:40:41.458496+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:41.458496+0000","last_undegraded":"2023-10-06T01:40:41.458496+0000","last_fullsized":"2023-10-06T01:40:41.458496+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480021+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:42.480021+0000","last_peered":"2023-10-06T01:40:42.480021+0000","last_clean":"2023-10-06T01:40:42.480021+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:42.480021+0000","last_undegraded":"2023-10-06T01:40:42.480021+0000","last_fullsized":"2023-10-06T01:40:42.480021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468486+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:40:41.468486+0000","last_peered":"2023-10-06T01:40:41.468486+0000","last_clean":"2023-10-06T01:40:41.468486+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:40:41.468486+0000","last_undegraded":"2023-10-06T01:40:41.468486+0000","last_fullsized":"2023-10-06T01:40:41.468486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456774+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:41.456774+0000","last_peered":"2023-10-06T01:40:41.456774+0000","last_clean":"2023-10-06T01:40:41.456774+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:41.456774+0000","last_undegraded":"2023-10-06T01:40:41.456774+0000","last_fullsized":"2023-10-06T01:40:41.456774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485111+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:42.485111+0000","last_peered":"2023-10-06T01:40:42.485111+0000","last_clean":"2023-10-06T01:40:42.485111+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:42.485111+0000","last_undegraded":"2023-10-06T01:40:42.485111+0000","last_fullsized":"2023-10-06T01:40:42.485111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.462919+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:40:41.462919+0000","last_peered":"2023-10-06T01:40:41.462919+0000","last_clean":"2023-10-06T01:40:41.462919+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:40:41.462919+0000","last_undegraded":"2023-10-06T01:40:41.462919+0000","last_fullsized":"2023-10-06T01:40:41.462919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456248+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:40:40.456248+0000","last_peered":"2023-10-06T01:40:40.456248+0000","last_clean":"2023-10-06T01:40:40.456248+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:40:40.456248+0000","last_undegraded":"2023-10-06T01:40:40.456248+0000","last_fullsized":"2023-10-06T01:40:40.456248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459939+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.461788+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:40:41.461788+0000","last_peered":"2023-10-06T01:40:41.461788+0000","last_clean":"2023-10-06T01:40:41.461788+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:40:41.461788+0000","last_undegraded":"2023-10-06T01:40:41.461788+0000","last_fullsized":"2023-10-06T01:40:41.461788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469064+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:41.469064+0000","last_peered":"2023-10-06T01:40:41.469064+0000","last_clean":"2023-10-06T01:40:41.469064+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:41.469064+0000","last_undegraded":"2023-10-06T01:40:41.469064+0000","last_fullsized":"2023-10-06T01:40:41.469064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455484+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471786+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:41.471786+0000","last_peered":"2023-10-06T01:40:41.471786+0000","last_clean":"2023-10-06T01:40:41.471786+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:41.471786+0000","last_undegraded":"2023-10-06T01:40:41.471786+0000","last_fullsized":"2023-10-06T01:40:41.471786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469790+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:41.469790+0000","last_peered":"2023-10-06T01:40:41.469790+0000","last_clean":"2023-10-06T01:40:41.469790+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:41.469790+0000","last_undegraded":"2023-10-06T01:40:41.469790+0000","last_fullsized":"2023-10-06T01:40:41.469790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":766304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297688,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074152,"statfs":{"total":96636764160,"available":96331931648,"internally_reserved":0,"allocated":409600,"data_stored":176490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":751850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.254},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":175880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":167057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.249},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.307},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.378}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.622}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.309},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.624}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":161482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.372},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34100000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:46.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.040+0000 7fa4f92a1700 1 Processor -- start 2023-10-06T01:40:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.040+0000 7fa4f92a1700 1 -- start start 2023-10-06T01:40:47.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f92a1700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4f40ad250 0x7fa4f40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f92a1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f92a1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f40ca640 con 0x7fa4f40ad620 2023-10-06T01:40:47.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f92a1700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40ced30 con 0x7fa4f40ad250 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f92a1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40cf010 con 0x7fa4f415db50 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f2ffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4f40ad250 0x7fa4f40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f2ffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4f40ad250 0x7fa4f40cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:47100/0 (socket says 172.21.15.98:47100) 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:57514/0 (socket says 172.21.15.98:57514) 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/2258857935 learned_addr learned my addr 172.21.15.98:0/2258857935 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.041+0000 7fa4ea7fc700 1 --1- 172.21.15.98:0/2258857935 >> v1:172.21.15.184:6789/0 conn(0x7fa4f40ad620 0x7fa4f415d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:40870/0 (socket says 172.21.15.98:40870) 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 msgr2=0x7fa4f40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f2ffd700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/2258857935 >> v1:172.21.15.184:6789/0 conn(0x7fa4f40ad620 legacy=0x7fa4f415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f27fc700 1 -- 172.21.15.98:0/2258857935 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998782628 0 0) 0x7fa4f40ca640 con 0x7fa4f40ad620 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/2258857935 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40ccad0 con 0x7fa4f40ad250 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f37fe700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f2ffd700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f40ad250 0x7fa4f40cd2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009990 tx=0x7fa4e4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fcc2add0af4d355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f27fc700 1 -- 172.21.15.98:0/2258857935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4009e80 con 0x7fa4f40ad250 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f27fc700 1 -- 172.21.15.98:0/2258857935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4005c10 con 0x7fa4f40ad250 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f27fc700 1 -- 172.21.15.98:0/2258857935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400b650 con 0x7fa4f40ad250 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f92a1700 1 -- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f40ad250 msgr2=0x7fa4f40cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.042+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f40ad250 0x7fa4f40cd2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009990 tx=0x7fa4e4009220 comp rx=0 tx=0).stop 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 -- 172.21.15.98:0/2258857935 shutdown_connections 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f415db50 0x7fa4f40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/2258857935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f40ad250 0x7fa4f40cd2d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 -- 172.21.15.98:0/2258857935 >> 172.21.15.98:0/2258857935 conn(0x7fa4f4067dd0 msgr2=0x7fa4f40ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 -- 172.21.15.98:0/2258857935 shutdown_connections 2023-10-06T01:40:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.043+0000 7fa4f92a1700 1 -- 172.21.15.98:0/2258857935 wait complete. 2023-10-06T01:40:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.044+0000 7fa4f92a1700 1 Processor -- start 2023-10-06T01:40:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.044+0000 7fa4f92a1700 1 -- start start 2023-10-06T01:40:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f40ad620 0x7fa4f40c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 0x7fa4f416dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40af2a0 con 0x7fa4f40ad250 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40af540 con 0x7fa4f415db50 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40af820 con 0x7fa4f40ad620 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46990/0 (socket says 172.21.15.98:46990) 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/3605189695 learned_addr learned my addr 172.21.15.98:0/3605189695 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4ea7fc700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f40ad620 0x7fa4f40c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f37fe700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 0x7fa4f416dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f40ad620 msgr2=0x7fa4f40c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f40ad620 0x7fa4f40c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 msgr2=0x7fa4f416dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 0x7fa4f416dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 -- 172.21.15.98:0/3605189695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec009a20 con 0x7fa4f40ad250 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f37fe700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 0x7fa4f416dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f2ffd700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40704c0 tx=0x7fa4e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e401dd90 con 0x7fa4f40ad250 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4026430 con 0x7fa4f40ad250 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40267d0 con 0x7fa4f40ad250 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e4009480 con 0x7fa4f40ad250 2023-10-06T01:40:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.045+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f40afde0 con 0x7fa4f40ad250 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.047+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f4064ad0 con 0x7fa4f40ad250 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.047+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4e401a420 con 0x7fa4f40ad250 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.047+0000 7fa4eb7fe700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 0x7fa4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.047+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa4e4086140 con 0x7fa4f40ad250 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.047+0000 7fa4ea7fc700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 0x7fa4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.048+0000 7fa4ea7fc700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 0x7fa4d40693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4f406d100 tx=0x7fa4dc007bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.051+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4e4053190 con 0x7fa4f40ad250 2023-10-06T01:40:47.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.228+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4f40ccdb0 con 0x7fa4d4066f30 2023-10-06T01:40:47.237 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.237+0000 7fa4eb7fe700 1 -- 172.21.15.98:0/3605189695 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310213 (secure 0 0 0) 0x7fa4f40ccdb0 con 0x7fa4d4066f30 2023-10-06T01:40:47.241 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 msgr2=0x7fa4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 0x7fa4d40693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4f406d100 tx=0x7fa4dc007bf0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 msgr2=0x7fa4f40b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40704c0 tx=0x7fa4e4016000 comp rx=0 tx=0).stop 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 shutdown_connections 2023-10-06T01:40:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa4d4066f30 0x7fa4d40693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4f415db50 0x7fa4f416dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4f40ad250 0x7fa4f40b05e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 --2- 172.21.15.98:0/3605189695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4f40ad620 0x7fa4f40c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 >> 172.21.15.98:0/3605189695 conn(0x7fa4f4067dd0 msgr2=0x7fa4f40692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 shutdown_connections 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.240+0000 7fa4f92a1700 1 -- 172.21.15.98:0/3605189695 wait complete. 2023-10-06T01:40:47.244 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:47.252 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T01:40:45.797527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382528,"kb_used_data":4224,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592192,"statfs":{"total":773094113280,"available":770654404608,"internally_reserved":0,"allocated":4325376,"data_stored":2535883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.349283"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478358+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:42.478358+0000","last_peered":"2023-10-06T01:40:42.478358+0000","last_clean":"2023-10-06T01:40:42.478358+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:42.478358+0000","last_undegraded":"2023-10-06T01:40:42.478358+0000","last_fullsized":"2023-10-06T01:40:42.478358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459724+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457109+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479027+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:42.479027+0000","last_peered":"2023-10-06T01:40:42.479027+0000","last_clean":"2023-10-06T01:40:42.479027+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:42.479027+0000","last_undegraded":"2023-10-06T01:40:42.479027+0000","last_fullsized":"2023-10-06T01:40:42.479027+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459074+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:06.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459386+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477100+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:42.477100+0000","last_peered":"2023-10-06T01:40:42.477100+0000","last_clean":"2023-10-06T01:40:42.477100+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:42.477100+0000","last_undegraded":"2023-10-06T01:40:42.477100+0000","last_fullsized":"2023-10-06T01:40:42.477100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479538+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:42.479538+0000","last_peered":"2023-10-06T01:40:42.479538+0000","last_clean":"2023-10-06T01:40:42.479538+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:42.479538+0000","last_undegraded":"2023-10-06T01:40:42.479538+0000","last_fullsized":"2023-10-06T01:40:42.479538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478626+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:42.478626+0000","last_peered":"2023-10-06T01:40:42.478626+0000","last_clean":"2023-10-06T01:40:42.478626+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:42.478626+0000","last_undegraded":"2023-10-06T01:40:42.478626+0000","last_fullsized":"2023-10-06T01:40:42.478626+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480200+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:42.480200+0000","last_peered":"2023-10-06T01:40:42.480200+0000","last_clean":"2023-10-06T01:40:42.480200+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:42.480200+0000","last_undegraded":"2023-10-06T01:40:42.480200+0000","last_fullsized":"2023-10-06T01:40:42.480200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481292+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:42.481292+0000","last_peered":"2023-10-06T01:40:42.481292+0000","last_clean":"2023-10-06T01:40:42.481292+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:42.481292+0000","last_undegraded":"2023-10-06T01:40:42.481292+0000","last_fullsized":"2023-10-06T01:40:42.481292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.460547+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457971+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:51.433736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480979+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:42.480979+0000","last_peered":"2023-10-06T01:40:42.480979+0000","last_clean":"2023-10-06T01:40:42.480979+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:42.480979+0000","last_undegraded":"2023-10-06T01:40:42.480979+0000","last_fullsized":"2023-10-06T01:40:42.480979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457286+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478135+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:42.478135+0000","last_peered":"2023-10-06T01:40:42.478135+0000","last_clean":"2023-10-06T01:40:42.478135+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:42.478135+0000","last_undegraded":"2023-10-06T01:40:42.478135+0000","last_fullsized":"2023-10-06T01:40:42.478135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455825+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456995+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482708+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:42.482708+0000","last_peered":"2023-10-06T01:40:42.482708+0000","last_clean":"2023-10-06T01:40:42.482708+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:42.482708+0000","last_undegraded":"2023-10-06T01:40:42.482708+0000","last_fullsized":"2023-10-06T01:40:42.482708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458747+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455840+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481147+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:42.481147+0000","last_peered":"2023-10-06T01:40:42.481147+0000","last_clean":"2023-10-06T01:40:42.481147+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:42.481147+0000","last_undegraded":"2023-10-06T01:40:42.481147+0000","last_fullsized":"2023-10-06T01:40:42.481147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.472954+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:42.472954+0000","last_peered":"2023-10-06T01:40:42.472954+0000","last_clean":"2023-10-06T01:40:42.472954+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:42.472954+0000","last_undegraded":"2023-10-06T01:40:42.472954+0000","last_fullsized":"2023-10-06T01:40:42.472954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473497+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:42.473497+0000","last_peered":"2023-10-06T01:40:42.473497+0000","last_clean":"2023-10-06T01:40:42.473497+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:42.473497+0000","last_undegraded":"2023-10-06T01:40:42.473497+0000","last_fullsized":"2023-10-06T01:40:42.473497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474680+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:42.474680+0000","last_peered":"2023-10-06T01:40:42.474680+0000","last_clean":"2023-10-06T01:40:42.474680+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:42.474680+0000","last_undegraded":"2023-10-06T01:40:42.474680+0000","last_fullsized":"2023-10-06T01:40:42.474680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.458345+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456575+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.463947+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478517+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:42.478517+0000","last_peered":"2023-10-06T01:40:42.478517+0000","last_clean":"2023-10-06T01:40:42.478517+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:42.478517+0000","last_undegraded":"2023-10-06T01:40:42.478517+0000","last_fullsized":"2023-10-06T01:40:42.478517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455323+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455997+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457381+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:52.900640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456256+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467749+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:40:41.467749+0000","last_peered":"2023-10-06T01:40:41.467749+0000","last_clean":"2023-10-06T01:40:41.467749+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:40:41.467749+0000","last_undegraded":"2023-10-06T01:40:41.467749+0000","last_fullsized":"2023-10-06T01:40:41.467749+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465919+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:42.465919+0000","last_peered":"2023-10-06T01:40:42.465919+0000","last_clean":"2023-10-06T01:40:42.465919+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:42.465919+0000","last_undegraded":"2023-10-06T01:40:42.465919+0000","last_fullsized":"2023-10-06T01:40:42.465919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.472783+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:40:41.472783+0000","last_peered":"2023-10-06T01:40:41.472783+0000","last_clean":"2023-10-06T01:40:41.472783+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:40:41.472783+0000","last_undegraded":"2023-10-06T01:40:41.472783+0000","last_fullsized":"2023-10-06T01:40:41.472783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.461022+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470447+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:40:41.470447+0000","last_peered":"2023-10-06T01:40:41.470447+0000","last_clean":"2023-10-06T01:40:41.470447+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:40:41.470447+0000","last_undegraded":"2023-10-06T01:40:41.470447+0000","last_fullsized":"2023-10-06T01:40:41.470447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455453+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484424+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:40:42.484424+0000","last_peered":"2023-10-06T01:40:42.484424+0000","last_clean":"2023-10-06T01:40:42.484424+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:40:42.484424+0000","last_undegraded":"2023-10-06T01:40:42.484424+0000","last_fullsized":"2023-10-06T01:40:42.484424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454760+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464311+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:40:41.464311+0000","last_peered":"2023-10-06T01:40:41.464311+0000","last_clean":"2023-10-06T01:40:41.464311+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:40:41.464311+0000","last_undegraded":"2023-10-06T01:40:41.464311+0000","last_fullsized":"2023-10-06T01:40:41.464311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475400+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:42.475400+0000","last_peered":"2023-10-06T01:40:42.475400+0000","last_clean":"2023-10-06T01:40:42.475400+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:42.475400+0000","last_undegraded":"2023-10-06T01:40:42.475400+0000","last_fullsized":"2023-10-06T01:40:42.475400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468752+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:40:41.468752+0000","last_peered":"2023-10-06T01:40:41.468752+0000","last_clean":"2023-10-06T01:40:41.468752+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:40:41.468752+0000","last_undegraded":"2023-10-06T01:40:41.468752+0000","last_fullsized":"2023-10-06T01:40:41.468752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478912+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:42.478912+0000","last_peered":"2023-10-06T01:40:42.478912+0000","last_clean":"2023-10-06T01:40:42.478912+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:42.478912+0000","last_undegraded":"2023-10-06T01:40:42.478912+0000","last_fullsized":"2023-10-06T01:40:42.478912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467878+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:40:41.467878+0000","last_peered":"2023-10-06T01:40:41.467878+0000","last_clean":"2023-10-06T01:40:41.467878+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:40:41.467878+0000","last_undegraded":"2023-10-06T01:40:41.467878+0000","last_fullsized":"2023-10-06T01:40:41.467878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473067+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:42.473067+0000","last_peered":"2023-10-06T01:40:42.473067+0000","last_clean":"2023-10-06T01:40:42.473067+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:42.473067+0000","last_undegraded":"2023-10-06T01:40:42.473067+0000","last_fullsized":"2023-10-06T01:40:42.473067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470737+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:41.470737+0000","last_peered":"2023-10-06T01:40:41.470737+0000","last_clean":"2023-10-06T01:40:41.470737+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:41.470737+0000","last_undegraded":"2023-10-06T01:40:41.470737+0000","last_fullsized":"2023-10-06T01:40:41.470737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456224+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.463319+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:40:41.463319+0000","last_peered":"2023-10-06T01:40:41.463319+0000","last_clean":"2023-10-06T01:40:41.463319+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:40:41.463319+0000","last_undegraded":"2023-10-06T01:40:41.463319+0000","last_fullsized":"2023-10-06T01:40:41.463319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483351+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:42.483351+0000","last_peered":"2023-10-06T01:40:42.483351+0000","last_clean":"2023-10-06T01:40:42.483351+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:42.483351+0000","last_undegraded":"2023-10-06T01:40:42.483351+0000","last_fullsized":"2023-10-06T01:40:42.483351+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471793+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:41.471793+0000","last_peered":"2023-10-06T01:40:41.471793+0000","last_clean":"2023-10-06T01:40:41.471793+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:41.471793+0000","last_undegraded":"2023-10-06T01:40:41.471793+0000","last_fullsized":"2023-10-06T01:40:41.471793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455708+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490915+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:40:42.490915+0000","last_peered":"2023-10-06T01:40:42.490915+0000","last_clean":"2023-10-06T01:40:42.490915+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:40:42.490915+0000","last_undegraded":"2023-10-06T01:40:42.490915+0000","last_fullsized":"2023-10-06T01:40:42.490915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478379+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:42.478379+0000","last_peered":"2023-10-06T01:40:42.478379+0000","last_clean":"2023-10-06T01:40:42.478379+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:42.478379+0000","last_undegraded":"2023-10-06T01:40:42.478379+0000","last_fullsized":"2023-10-06T01:40:42.478379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465691+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:40:41.465691+0000","last_peered":"2023-10-06T01:40:41.465691+0000","last_clean":"2023-10-06T01:40:41.465691+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:40:41.465691+0000","last_undegraded":"2023-10-06T01:40:41.465691+0000","last_fullsized":"2023-10-06T01:40:41.465691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484065+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:42.484065+0000","last_peered":"2023-10-06T01:40:42.484065+0000","last_clean":"2023-10-06T01:40:42.484065+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:42.484065+0000","last_undegraded":"2023-10-06T01:40:42.484065+0000","last_fullsized":"2023-10-06T01:40:42.484065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468974+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:40:41.468974+0000","last_peered":"2023-10-06T01:40:41.468974+0000","last_clean":"2023-10-06T01:40:41.468974+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:40:41.468974+0000","last_undegraded":"2023-10-06T01:40:41.468974+0000","last_fullsized":"2023-10-06T01:40:41.468974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478175+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:42.478175+0000","last_peered":"2023-10-06T01:40:42.478175+0000","last_clean":"2023-10-06T01:40:42.478175+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:42.478175+0000","last_undegraded":"2023-10-06T01:40:42.478175+0000","last_fullsized":"2023-10-06T01:40:42.478175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469395+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:41.469395+0000","last_peered":"2023-10-06T01:40:41.469395+0000","last_clean":"2023-10-06T01:40:41.469395+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:41.469395+0000","last_undegraded":"2023-10-06T01:40:41.469395+0000","last_fullsized":"2023-10-06T01:40:41.469395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455404+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.467708+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:40:41.467708+0000","last_peered":"2023-10-06T01:40:41.467708+0000","last_clean":"2023-10-06T01:40:41.467708+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:40:41.467708+0000","last_undegraded":"2023-10-06T01:40:41.467708+0000","last_fullsized":"2023-10-06T01:40:41.467708+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481908+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:42.481908+0000","last_peered":"2023-10-06T01:40:42.481908+0000","last_clean":"2023-10-06T01:40:42.481908+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:42.481908+0000","last_undegraded":"2023-10-06T01:40:42.481908+0000","last_fullsized":"2023-10-06T01:40:42.481908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491752+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:40:42.491752+0000","last_peered":"2023-10-06T01:40:42.491752+0000","last_clean":"2023-10-06T01:40:42.491752+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:40:42.491752+0000","last_undegraded":"2023-10-06T01:40:42.491752+0000","last_fullsized":"2023-10-06T01:40:42.491752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.457872+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:03.458529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.466682+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:40:41.466682+0000","last_peered":"2023-10-06T01:40:41.466682+0000","last_clean":"2023-10-06T01:40:41.466682+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:40:41.466682+0000","last_undegraded":"2023-10-06T01:40:41.466682+0000","last_fullsized":"2023-10-06T01:40:41.466682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483508+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:42.483508+0000","last_peered":"2023-10-06T01:40:42.483508+0000","last_clean":"2023-10-06T01:40:42.483508+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:42.483508+0000","last_undegraded":"2023-10-06T01:40:42.483508+0000","last_fullsized":"2023-10-06T01:40:42.483508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.465545+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:40:41.465545+0000","last_peered":"2023-10-06T01:40:41.465545+0000","last_clean":"2023-10-06T01:40:41.465545+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:40:41.465545+0000","last_undegraded":"2023-10-06T01:40:41.465545+0000","last_fullsized":"2023-10-06T01:40:41.465545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474713+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:42.474713+0000","last_peered":"2023-10-06T01:40:42.474713+0000","last_clean":"2023-10-06T01:40:42.474713+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:42.474713+0000","last_undegraded":"2023-10-06T01:40:42.474713+0000","last_fullsized":"2023-10-06T01:40:42.474713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468411+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:40:41.468411+0000","last_peered":"2023-10-06T01:40:41.468411+0000","last_clean":"2023-10-06T01:40:41.468411+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:40:41.468411+0000","last_undegraded":"2023-10-06T01:40:41.468411+0000","last_fullsized":"2023-10-06T01:40:41.468411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.471507+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:42.471507+0000","last_peered":"2023-10-06T01:40:42.471507+0000","last_clean":"2023-10-06T01:40:42.471507+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:42.471507+0000","last_undegraded":"2023-10-06T01:40:42.471507+0000","last_fullsized":"2023-10-06T01:40:42.471507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468147+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:40:41.468147+0000","last_peered":"2023-10-06T01:40:41.468147+0000","last_clean":"2023-10-06T01:40:41.468147+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:40:41.468147+0000","last_undegraded":"2023-10-06T01:40:41.468147+0000","last_fullsized":"2023-10-06T01:40:41.468147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456078+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464033+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:40:41.464033+0000","last_peered":"2023-10-06T01:40:41.464033+0000","last_clean":"2023-10-06T01:40:41.464033+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:40:41.464033+0000","last_undegraded":"2023-10-06T01:40:41.464033+0000","last_fullsized":"2023-10-06T01:40:41.464033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455168+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.473889+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:40:41.473889+0000","last_peered":"2023-10-06T01:40:41.473889+0000","last_clean":"2023-10-06T01:40:41.473889+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:40:41.473889+0000","last_undegraded":"2023-10-06T01:40:41.473889+0000","last_fullsized":"2023-10-06T01:40:41.473889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.456925+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:23.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.474303+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:41.474303+0000","last_peered":"2023-10-06T01:40:41.474303+0000","last_clean":"2023-10-06T01:40:41.474303+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:41.474303+0000","last_undegraded":"2023-10-06T01:40:41.474303+0000","last_fullsized":"2023-10-06T01:40:41.474303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478522+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:42.478522+0000","last_peered":"2023-10-06T01:40:42.478522+0000","last_clean":"2023-10-06T01:40:42.478522+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:42.478522+0000","last_undegraded":"2023-10-06T01:40:42.478522+0000","last_fullsized":"2023-10-06T01:40:42.478522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469403+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:41.469403+0000","last_peered":"2023-10-06T01:40:41.469403+0000","last_clean":"2023-10-06T01:40:41.469403+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:41.469403+0000","last_undegraded":"2023-10-06T01:40:41.469403+0000","last_fullsized":"2023-10-06T01:40:41.469403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454931+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.464545+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:40:41.464545+0000","last_peered":"2023-10-06T01:40:41.464545+0000","last_clean":"2023-10-06T01:40:41.464545+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:40:41.464545+0000","last_undegraded":"2023-10-06T01:40:41.464545+0000","last_fullsized":"2023-10-06T01:40:41.464545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456794+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:40:40.456794+0000","last_peered":"2023-10-06T01:40:40.456794+0000","last_clean":"2023-10-06T01:40:40.456794+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:40:40.456794+0000","last_undegraded":"2023-10-06T01:40:40.456794+0000","last_fullsized":"2023-10-06T01:40:40.456794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.454856+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477701+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:41.477701+0000","last_peered":"2023-10-06T01:40:41.477701+0000","last_clean":"2023-10-06T01:40:41.477701+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:41.477701+0000","last_undegraded":"2023-10-06T01:40:41.477701+0000","last_fullsized":"2023-10-06T01:40:41.477701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454750+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:40:40.454750+0000","last_peered":"2023-10-06T01:40:40.454750+0000","last_clean":"2023-10-06T01:40:40.454750+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:40:40.454750+0000","last_undegraded":"2023-10-06T01:40:40.454750+0000","last_fullsized":"2023-10-06T01:40:40.454750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475832+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:42.475832+0000","last_peered":"2023-10-06T01:40:42.475832+0000","last_clean":"2023-10-06T01:40:42.475832+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:42.475832+0000","last_undegraded":"2023-10-06T01:40:42.475832+0000","last_fullsized":"2023-10-06T01:40:42.475832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470594+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:40:41.470594+0000","last_peered":"2023-10-06T01:40:41.470594+0000","last_clean":"2023-10-06T01:40:41.470594+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:40:41.470594+0000","last_undegraded":"2023-10-06T01:40:41.470594+0000","last_fullsized":"2023-10-06T01:40:41.470594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.454576+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:40:40.454576+0000","last_peered":"2023-10-06T01:40:40.454576+0000","last_clean":"2023-10-06T01:40:40.454576+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:40:40.454576+0000","last_undegraded":"2023-10-06T01:40:40.454576+0000","last_fullsized":"2023-10-06T01:40:40.454576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456861+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:41.456861+0000","last_peered":"2023-10-06T01:40:41.456861+0000","last_clean":"2023-10-06T01:40:41.456861+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:41.456861+0000","last_undegraded":"2023-10-06T01:40:41.456861+0000","last_fullsized":"2023-10-06T01:40:41.456861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479190+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:42.479190+0000","last_peered":"2023-10-06T01:40:42.479190+0000","last_clean":"2023-10-06T01:40:42.479190+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:42.479190+0000","last_undegraded":"2023-10-06T01:40:42.479190+0000","last_fullsized":"2023-10-06T01:40:42.479190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477681+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:41.477681+0000","last_peered":"2023-10-06T01:40:41.477681+0000","last_clean":"2023-10-06T01:40:41.477681+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:41.477681+0000","last_undegraded":"2023-10-06T01:40:41.477681+0000","last_fullsized":"2023-10-06T01:40:41.477681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.458496+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:41.458496+0000","last_peered":"2023-10-06T01:40:41.458496+0000","last_clean":"2023-10-06T01:40:41.458496+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:41.458496+0000","last_undegraded":"2023-10-06T01:40:41.458496+0000","last_fullsized":"2023-10-06T01:40:41.458496+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480021+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:42.480021+0000","last_peered":"2023-10-06T01:40:42.480021+0000","last_clean":"2023-10-06T01:40:42.480021+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:42.480021+0000","last_undegraded":"2023-10-06T01:40:42.480021+0000","last_fullsized":"2023-10-06T01:40:42.480021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.468486+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:40:41.468486+0000","last_peered":"2023-10-06T01:40:41.468486+0000","last_clean":"2023-10-06T01:40:41.468486+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:40:41.468486+0000","last_undegraded":"2023-10-06T01:40:41.468486+0000","last_fullsized":"2023-10-06T01:40:41.468486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456774+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:41.456774+0000","last_peered":"2023-10-06T01:40:41.456774+0000","last_clean":"2023-10-06T01:40:41.456774+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:41.456774+0000","last_undegraded":"2023-10-06T01:40:41.456774+0000","last_fullsized":"2023-10-06T01:40:41.456774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485111+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:42.485111+0000","last_peered":"2023-10-06T01:40:42.485111+0000","last_clean":"2023-10-06T01:40:42.485111+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:42.485111+0000","last_undegraded":"2023-10-06T01:40:42.485111+0000","last_fullsized":"2023-10-06T01:40:42.485111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.462919+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:40:41.462919+0000","last_peered":"2023-10-06T01:40:41.462919+0000","last_clean":"2023-10-06T01:40:41.462919+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:40:41.462919+0000","last_undegraded":"2023-10-06T01:40:41.462919+0000","last_fullsized":"2023-10-06T01:40:41.462919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T01:40:40.456248+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:40:40.456248+0000","last_peered":"2023-10-06T01:40:40.456248+0000","last_clean":"2023-10-06T01:40:40.456248+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:40:40.456248+0000","last_undegraded":"2023-10-06T01:40:40.456248+0000","last_fullsized":"2023-10-06T01:40:40.456248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.459939+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.461788+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:40:41.461788+0000","last_peered":"2023-10-06T01:40:41.461788+0000","last_clean":"2023-10-06T01:40:41.461788+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:40:41.461788+0000","last_undegraded":"2023-10-06T01:40:41.461788+0000","last_fullsized":"2023-10-06T01:40:41.461788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469064+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:41.469064+0000","last_peered":"2023-10-06T01:40:41.469064+0000","last_clean":"2023-10-06T01:40:41.469064+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:41.469064+0000","last_undegraded":"2023-10-06T01:40:41.469064+0000","last_fullsized":"2023-10-06T01:40:41.469064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T01:40:41.447997+0000","last_change":"2023-10-06T01:40:41.455484+0000","last_active":"2023-10-06T01:40:41.447997+0000","last_peered":"2023-10-06T01:40:41.447997+0000","last_clean":"2023-10-06T01:40:41.447997+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:40:41.447997+0000","last_undegraded":"2023-10-06T01:40:41.447997+0000","last_fullsized":"2023-10-06T01:40:41.447997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471786+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:41.471786+0000","last_peered":"2023-10-06T01:40:41.471786+0000","last_clean":"2023-10-06T01:40:41.471786+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:41.471786+0000","last_undegraded":"2023-10-06T01:40:41.471786+0000","last_fullsized":"2023-10-06T01:40:41.471786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469790+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:41.469790+0000","last_peered":"2023-10-06T01:40:41.469790+0000","last_clean":"2023-10-06T01:40:41.469790+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:41.469790+0000","last_undegraded":"2023-10-06T01:40:41.469790+0000","last_fullsized":"2023-10-06T01:40:41.469790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":766304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297688,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074152,"statfs":{"total":96636764160,"available":96331931648,"internally_reserved":0,"allocated":409600,"data_stored":176490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":751850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.254},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":175880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":167057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.249},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.307},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.378}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.622}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.309},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":168410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.624}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":161482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.372},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34100000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:47.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-10-06T01:40:47.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.459724+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.457109+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.459074+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:36:06.462696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.459386+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-10-06T01:40:47.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.460547+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.457971+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:26:51.433736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.457286+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455825+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-10-06T01:40:47.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456995+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.458747+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455840+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.458345+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456575+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.463947+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455323+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455997+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.457381+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:23:52.900640+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-10-06T01:40:47.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456256+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.461022+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455453+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.454760+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-10-06T01:40:47.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456224+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455708+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455404+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.457872+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:07:03.458529+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456078+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455168+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.456925+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:23.190744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.454931+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.454856+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T01:40:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-10-06T01:40:47.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.459939+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T01:40:47.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-10-06T01:40:47.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T01:40:41.447997+0000', 'last_change': '2023-10-06T01:40:41.455484+0000', 'last_active': '2023-10-06T01:40:41.447997+0000', 'last_peered': '2023-10-06T01:40:41.447997+0000', 'last_clean': '2023-10-06T01:40:41.447997+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T01:40:41.447997+0000', 'last_undegraded': '2023-10-06T01:40:41.447997+0000', 'last_fullsized': '2023-10-06T01:40:41.447997+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'last_clean_scrub_stamp': '2023-10-06T01:40:41.447997+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T01:40:47.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T01:40:47.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.405+0000 7f63f82b3700 1 Processor -- start 2023-10-06T01:40:47.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.405+0000 7f63f82b3700 1 -- start start 2023-10-06T01:40:47.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.406+0000 7f63f82b3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63f00b2620 0x7f63f00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.406+0000 7f63f82b3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f0159870 0x7f63f00cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.406+0000 7f63f82b3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00cbfe0 con 0x7f63f00b29f0 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.406+0000 7f63f82b3700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f00cb3d0 con 0x7f63f00b2620 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.406+0000 7f63f82b3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00cb6b0 con 0x7f63f0159870 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63f00b2620 0x7f63f00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63f00b2620 0x7f63f00b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46994/0 (socket says 172.21.15.98:46994) 2023-10-06T01:40:47.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 -- 172.21.15.98:0/3343517309 learned_addr learned my addr 172.21.15.98:0/3343517309 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f6850700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f0159870 0x7f63f00cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 -- 172.21.15.98:0/3343517309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f0159870 msgr2=0x7f63f00cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f0159870 0x7f63f00cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 -- 172.21.15.98:0/3343517309 >> v1:172.21.15.184:6789/0 conn(0x7f63f00b29f0 legacy=0x7f63f0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 -- 172.21.15.98:0/3343517309 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00cb990 con 0x7f63f00b2620 2023-10-06T01:40:47.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f604f700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f00b2620 0x7f63f00b06d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009f70 tx=0x7f63ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280476e95a2446b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f504d700 1 -- 172.21.15.98:0/3343517309 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec00e070 con 0x7f63f00b2620 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f504d700 1 -- 172.21.15.98:0/3343517309 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec00ba90 con 0x7f63f00b2620 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.407+0000 7f63f504d700 1 -- 172.21.15.98:0/3343517309 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0092e0 con 0x7f63f00b2620 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 -- 172.21.15.98:0/3343517309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f00b2620 msgr2=0x7f63f00b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f00b2620 0x7f63f00b06d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009f70 tx=0x7f63ec009840 comp rx=0 tx=0).stop 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 -- 172.21.15.98:0/3343517309 shutdown_connections 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f0159870 0x7f63f00cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 --2- 172.21.15.98:0/3343517309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f00b2620 0x7f63f00b06d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 -- 172.21.15.98:0/3343517309 >> 172.21.15.98:0/3343517309 conn(0x7f63f006ed20 msgr2=0x7f63f006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:47.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 -- 172.21.15.98:0/3343517309 shutdown_connections 2023-10-06T01:40:47.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.408+0000 7f63f82b3700 1 -- 172.21.15.98:0/3343517309 wait complete. 2023-10-06T01:40:47.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.409+0000 7f63f82b3700 1 Processor -- start 2023-10-06T01:40:47.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.409+0000 7f63f82b3700 1 -- start start 2023-10-06T01:40:47.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f00b2620 0x7f63f01447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63f00b29f0 0x7f63f015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0169d60 con 0x7f63f0159870 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00c8460 con 0x7f63f00b29f0 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0169f50 con 0x7f63f00b2620 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47008/0 (socket says 172.21.15.98:47008) 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 -- 172.21.15.98:0/2095003979 learned_addr learned my addr 172.21.15.98:0/2095003979 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:47.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f584e700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63f00b29f0 0x7f63f015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f604f700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f00b2620 0x7f63f01447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 -- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f00b2620 msgr2=0x7f63f01447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f00b2620 0x7f63f01447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 -- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63f00b29f0 msgr2=0x7f63f015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63f00b29f0 0x7f63f015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 -- 172.21.15.98:0/2095003979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec009ae0 con 0x7f63f0159870 2023-10-06T01:40:47.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f6850700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c880 tx=0x7f63e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8011600 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8011760 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f016a230 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8011b20 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.410+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f00ca860 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.411+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0064ad0 con 0x7f63f0159870 2023-10-06T01:40:47.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.412+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63e8018360 con 0x7f63f0159870 2023-10-06T01:40:47.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.412+0000 7f63e67fc700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 0x7f63d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.412+0000 7f63f604f700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 0x7f63d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.412+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f63e8084360 con 0x7f63f0159870 2023-10-06T01:40:47.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.413+0000 7f63f604f700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 0x7f63d40693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009880 tx=0x7f63ec017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.417+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63e8054dc0 con 0x7f63f0159870 2023-10-06T01:40:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.677+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f63f00cbc70 con 0x7f63f0159870 2023-10-06T01:40:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.677+0000 7f63e67fc700 1 -- 172.21.15.98:0/2095003979 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1647 (secure 0 0 0) 0x7f63e8054950 con 0x7f63f0159870 2023-10-06T01:40:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1696556377,"num_in_osds":8,"osd_in_since":1696556368,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":73},{"state_name":"creating+peering","count":32}],"num_pgs":105,"num_pools":4,"num_objects":9,"data_bytes":590458,"bytes_used":2439708672,"bytes_avail":770654404608,"bytes_total":773094113280,"inactive_pgs_ratio":0.30476191639900208,"write_bytes_sec":191,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":5133}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-06T01:40:39.796608+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T01:40:47.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 msgr2=0x7f63d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 0x7f63d40693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009880 tx=0x7f63ec017000 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 msgr2=0x7f63f015d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c880 tx=0x7f63e8008040 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 shutdown_connections 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63d4066f30 0x7f63d40693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63f0159870 0x7f63f015d770 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63f00b2620 0x7f63f01447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 --2- 172.21.15.98:0/2095003979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63f00b29f0 0x7f63f015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 >> 172.21.15.98:0/2095003979 conn(0x7f63f006ed20 msgr2=0x7f63f00c6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 shutdown_connections 2023-10-06T01:40:47.681 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.680+0000 7f63f82b3700 1 -- 172.21.15.98:0/2095003979 wait complete. 2023-10-06T01:40:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T01:40:47.692 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:47.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.805+0000 7f4ba4911700 1 Processor -- start 2023-10-06T01:40:47.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.805+0000 7f4ba4911700 1 -- start start 2023-10-06T01:40:47.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4ba4911700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 0x7f4ba00c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4ba4911700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba00acd60 con 0x7f4ba00c75b0 2023-10-06T01:40:47.814 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4ba4911700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba0156910 con 0x7f4ba006eda0 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4ba4911700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba006f470 con 0x7f4ba0156540 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4b95d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4ba00c75b0 0x7f4ba006e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45658/0 (socket says 172.21.15.98:45658) 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4b95d9b700 1 -- 172.21.15.98:0/881482671 learned_addr learned my addr 172.21.15.98:0/881482671 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.806+0000 7f4b9e59c700 1 --2- 172.21.15.98:0/881482671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 0x7f4ba00c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655627421 0 0) 0x7f4ba00acd60 con 0x7f4ba00c75b0 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8c003570 con 0x7f4ba00c75b0 2023-10-06T01:40:47.815 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521960336 0 0) 0x7f4ba0156910 con 0x7f4ba006eda0 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba00acd60 con 0x7f4ba006eda0 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715925962 0 0) 0x7f4b8c003570 con 0x7f4ba00c75b0 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9e59c700 1 -- 172.21.15.98:0/881482671 >> v1:172.21.15.184:6789/0 conn(0x7f4ba006eda0 legacy=0x7f4ba00ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9e59c700 1 -- 172.21.15.98:0/881482671 >> v1:172.21.15.98:6789/0 conn(0x7f4ba00c75b0 legacy=0x7f4ba006e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9e59c700 1 -- 172.21.15.98:0/881482671 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba00c7b70 con 0x7f4ba0156540 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9e59c700 1 --2- 172.21.15.98:0/881482671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 0x7f4ba00c7070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009910 tx=0x7f4b900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f82e7eb988636ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.816 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88001470 con 0x7f4ba0156540 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90011420 con 0x7f4ba0156540 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.807+0000 7f4b9dd9b700 1 -- 172.21.15.98:0/881482671 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900117a0 con 0x7f4ba0156540 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 -- 172.21.15.98:0/881482671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 msgr2=0x7f4ba00c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 --2- 172.21.15.98:0/881482671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 0x7f4ba00c7070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009910 tx=0x7f4b900091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 -- 172.21.15.98:0/881482671 shutdown_connections 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 --2- 172.21.15.98:0/881482671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba0156540 0x7f4ba00c7070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 -- 172.21.15.98:0/881482671 >> 172.21.15.98:0/881482671 conn(0x7f4ba00680c0 msgr2=0x7f4ba00cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:47.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 -- 172.21.15.98:0/881482671 shutdown_connections 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.808+0000 7f4ba4911700 1 -- 172.21.15.98:0/881482671 wait complete. 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.809+0000 7f4ba4911700 1 Processor -- start 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.809+0000 7f4ba4911700 1 -- start start 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba006eda0 0x7f4ba00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 0x7f4ba01541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0169b80 con 0x7f4ba0156540 2023-10-06T01:40:47.818 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00cb9e0 con 0x7f4ba00c75b0 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0154720 con 0x7f4ba006eda0 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47024/0 (socket says 172.21.15.98:47024) 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 -- 172.21.15.98:0/159429729 learned_addr learned my addr 172.21.15.98:0/159429729 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9e59c700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba006eda0 0x7f4ba00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 -- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba006eda0 msgr2=0x7f4ba00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b95d9b700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 0x7f4ba01541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.819 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba006eda0 0x7f4ba00cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 -- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 msgr2=0x7f4ba01541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 0x7f4ba01541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 -- 172.21.15.98:0/159429729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009420 con 0x7f4ba0156540 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b95d9b700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 0x7f4ba01541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b9ed9d700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c940 tx=0x7f4b98002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800ce80 con 0x7f4ba0156540 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800a7a0 con 0x7f4ba0156540 2023-10-06T01:40:47.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba00aa7b0 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.810+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800ab60 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.811+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba00a9e80 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.811+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba0064ad0 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.812+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b98012420 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.812+0000 7f4b96ffd700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 0x7f4b800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.812+0000 7f4b9e59c700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 0x7f4b800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.812+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4b98084b90 con 0x7f4ba0156540 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.813+0000 7f4b9e59c700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 0x7f4b800693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b90003080 tx=0x7f4b90016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:47.821 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.817+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b98051c10 con 0x7f4ba0156540 2023-10-06T01:40:47.998 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:47.994+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ba006bef0 con 0x7f4b80066f30 2023-10-06T01:40:48.003 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.002+0000 7f4b96ffd700 1 -- 172.21.15.98:0/159429729 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311692 (secure 0 0 0) 0x7f4b90011740 con 0x7f4b80066f30 2023-10-06T01:40:48.006 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 msgr2=0x7f4b800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 0x7f4b800693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b90003080 tx=0x7f4b90016000 comp rx=0 tx=0).stop 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 msgr2=0x7f4ba00a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c940 tx=0x7f4b98002720 comp rx=0 tx=0).stop 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 shutdown_connections 2023-10-06T01:40:48.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b80066f30 0x7f4b800693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ba0156540 0x7f4ba00a9410 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ba006eda0 0x7f4ba00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 --2- 172.21.15.98:0/159429729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ba00c75b0 0x7f4ba01541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 >> 172.21.15.98:0/159429729 conn(0x7f4ba00680c0 msgr2=0x7f4ba01557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 shutdown_connections 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:48.006+0000 7f4ba4911700 1 -- 172.21.15.98:0/159429729 wait complete. 2023-10-06T01:40:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:48.019 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-10-06T01:40:47.797849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382720,"kb_used_data":4416,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592000,"statfs":{"total":773094113280,"available":770654208000,"internally_reserved":0,"allocated":4521984,"data_stored":2625568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.346730"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478358+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:42.478358+0000","last_peered":"2023-10-06T01:40:42.478358+0000","last_clean":"2023-10-06T01:40:42.478358+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:42.478358+0000","last_undegraded":"2023-10-06T01:40:42.478358+0000","last_fullsized":"2023-10-06T01:40:42.478358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491145+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:40:42.491145+0000","last_peered":"2023-10-06T01:40:42.491145+0000","last_clean":"2023-10-06T01:40:42.491145+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:40:42.491145+0000","last_undegraded":"2023-10-06T01:40:42.491145+0000","last_fullsized":"2023-10-06T01:40:42.491145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478857+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:40:42.478857+0000","last_peered":"2023-10-06T01:40:42.478857+0000","last_clean":"2023-10-06T01:40:42.478857+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:40:42.478857+0000","last_undegraded":"2023-10-06T01:40:42.478857+0000","last_fullsized":"2023-10-06T01:40:42.478857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479027+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:42.479027+0000","last_peered":"2023-10-06T01:40:42.479027+0000","last_clean":"2023-10-06T01:40:42.479027+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:42.479027+0000","last_undegraded":"2023-10-06T01:40:42.479027+0000","last_fullsized":"2023-10-06T01:40:42.479027+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491125+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:40:42.491125+0000","last_peered":"2023-10-06T01:40:42.491125+0000","last_clean":"2023-10-06T01:40:42.491125+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:40:42.491125+0000","last_undegraded":"2023-10-06T01:40:42.491125+0000","last_fullsized":"2023-10-06T01:40:42.491125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484825+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:40:42.484825+0000","last_peered":"2023-10-06T01:40:42.484825+0000","last_clean":"2023-10-06T01:40:42.484825+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:40:42.484825+0000","last_undegraded":"2023-10-06T01:40:42.484825+0000","last_fullsized":"2023-10-06T01:40:42.484825+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477100+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:42.477100+0000","last_peered":"2023-10-06T01:40:42.477100+0000","last_clean":"2023-10-06T01:40:42.477100+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:42.477100+0000","last_undegraded":"2023-10-06T01:40:42.477100+0000","last_fullsized":"2023-10-06T01:40:42.477100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479538+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:42.479538+0000","last_peered":"2023-10-06T01:40:42.479538+0000","last_clean":"2023-10-06T01:40:42.479538+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:42.479538+0000","last_undegraded":"2023-10-06T01:40:42.479538+0000","last_fullsized":"2023-10-06T01:40:42.479538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478626+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:42.478626+0000","last_peered":"2023-10-06T01:40:42.478626+0000","last_clean":"2023-10-06T01:40:42.478626+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:42.478626+0000","last_undegraded":"2023-10-06T01:40:42.478626+0000","last_fullsized":"2023-10-06T01:40:42.478626+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480200+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:42.480200+0000","last_peered":"2023-10-06T01:40:42.480200+0000","last_clean":"2023-10-06T01:40:42.480200+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:42.480200+0000","last_undegraded":"2023-10-06T01:40:42.480200+0000","last_fullsized":"2023-10-06T01:40:42.480200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481292+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:42.481292+0000","last_peered":"2023-10-06T01:40:42.481292+0000","last_clean":"2023-10-06T01:40:42.481292+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:42.481292+0000","last_undegraded":"2023-10-06T01:40:42.481292+0000","last_fullsized":"2023-10-06T01:40:42.481292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491590+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:40:42.491590+0000","last_peered":"2023-10-06T01:40:42.491590+0000","last_clean":"2023-10-06T01:40:42.491590+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:40:42.491590+0000","last_undegraded":"2023-10-06T01:40:42.491590+0000","last_fullsized":"2023-10-06T01:40:42.491590+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482369+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:40:42.482369+0000","last_peered":"2023-10-06T01:40:42.482369+0000","last_clean":"2023-10-06T01:40:42.482369+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:40:42.482369+0000","last_undegraded":"2023-10-06T01:40:42.482369+0000","last_fullsized":"2023-10-06T01:40:42.482369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480979+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:42.480979+0000","last_peered":"2023-10-06T01:40:42.480979+0000","last_clean":"2023-10-06T01:40:42.480979+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:42.480979+0000","last_undegraded":"2023-10-06T01:40:42.480979+0000","last_fullsized":"2023-10-06T01:40:42.480979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484796+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:40:42.484796+0000","last_peered":"2023-10-06T01:40:42.484796+0000","last_clean":"2023-10-06T01:40:42.484796+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:40:42.484796+0000","last_undegraded":"2023-10-06T01:40:42.484796+0000","last_fullsized":"2023-10-06T01:40:42.484796+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478135+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:42.478135+0000","last_peered":"2023-10-06T01:40:42.478135+0000","last_clean":"2023-10-06T01:40:42.478135+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:42.478135+0000","last_undegraded":"2023-10-06T01:40:42.478135+0000","last_fullsized":"2023-10-06T01:40:42.478135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490826+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:40:42.490826+0000","last_peered":"2023-10-06T01:40:42.490826+0000","last_clean":"2023-10-06T01:40:42.490826+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:40:42.490826+0000","last_undegraded":"2023-10-06T01:40:42.490826+0000","last_fullsized":"2023-10-06T01:40:42.490826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487013+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:40:42.487013+0000","last_peered":"2023-10-06T01:40:42.487013+0000","last_clean":"2023-10-06T01:40:42.487013+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:40:42.487013+0000","last_undegraded":"2023-10-06T01:40:42.487013+0000","last_fullsized":"2023-10-06T01:40:42.487013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482708+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:42.482708+0000","last_peered":"2023-10-06T01:40:42.482708+0000","last_clean":"2023-10-06T01:40:42.482708+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:42.482708+0000","last_undegraded":"2023-10-06T01:40:42.482708+0000","last_fullsized":"2023-10-06T01:40:42.482708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488301+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:40:42.488301+0000","last_peered":"2023-10-06T01:40:42.488301+0000","last_clean":"2023-10-06T01:40:42.488301+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:40:42.488301+0000","last_undegraded":"2023-10-06T01:40:42.488301+0000","last_fullsized":"2023-10-06T01:40:42.488301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484798+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:40:42.484798+0000","last_peered":"2023-10-06T01:40:42.484798+0000","last_clean":"2023-10-06T01:40:42.484798+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:40:42.484798+0000","last_undegraded":"2023-10-06T01:40:42.484798+0000","last_fullsized":"2023-10-06T01:40:42.484798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481147+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:42.481147+0000","last_peered":"2023-10-06T01:40:42.481147+0000","last_clean":"2023-10-06T01:40:42.481147+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:42.481147+0000","last_undegraded":"2023-10-06T01:40:42.481147+0000","last_fullsized":"2023-10-06T01:40:42.481147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.472954+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:42.472954+0000","last_peered":"2023-10-06T01:40:42.472954+0000","last_clean":"2023-10-06T01:40:42.472954+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:42.472954+0000","last_undegraded":"2023-10-06T01:40:42.472954+0000","last_fullsized":"2023-10-06T01:40:42.472954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473497+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:42.473497+0000","last_peered":"2023-10-06T01:40:42.473497+0000","last_clean":"2023-10-06T01:40:42.473497+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:42.473497+0000","last_undegraded":"2023-10-06T01:40:42.473497+0000","last_fullsized":"2023-10-06T01:40:42.473497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474680+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:42.474680+0000","last_peered":"2023-10-06T01:40:42.474680+0000","last_clean":"2023-10-06T01:40:42.474680+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:42.474680+0000","last_undegraded":"2023-10-06T01:40:42.474680+0000","last_fullsized":"2023-10-06T01:40:42.474680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481527+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:40:42.481527+0000","last_peered":"2023-10-06T01:40:42.481527+0000","last_clean":"2023-10-06T01:40:42.481527+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:40:42.481527+0000","last_undegraded":"2023-10-06T01:40:42.481527+0000","last_fullsized":"2023-10-06T01:40:42.481527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490786+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:40:42.490786+0000","last_peered":"2023-10-06T01:40:42.490786+0000","last_clean":"2023-10-06T01:40:42.490786+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:40:42.490786+0000","last_undegraded":"2023-10-06T01:40:42.490786+0000","last_fullsized":"2023-10-06T01:40:42.490786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491609+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:40:42.491609+0000","last_peered":"2023-10-06T01:40:42.491609+0000","last_clean":"2023-10-06T01:40:42.491609+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:40:42.491609+0000","last_undegraded":"2023-10-06T01:40:42.491609+0000","last_fullsized":"2023-10-06T01:40:42.491609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478517+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:42.478517+0000","last_peered":"2023-10-06T01:40:42.478517+0000","last_clean":"2023-10-06T01:40:42.478517+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:42.478517+0000","last_undegraded":"2023-10-06T01:40:42.478517+0000","last_fullsized":"2023-10-06T01:40:42.478517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478891+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:40:42.478891+0000","last_peered":"2023-10-06T01:40:42.478891+0000","last_clean":"2023-10-06T01:40:42.478891+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:40:42.478891+0000","last_undegraded":"2023-10-06T01:40:42.478891+0000","last_fullsized":"2023-10-06T01:40:42.478891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484815+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:40:42.484815+0000","last_peered":"2023-10-06T01:40:42.484815+0000","last_clean":"2023-10-06T01:40:42.484815+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:40:42.484815+0000","last_undegraded":"2023-10-06T01:40:42.484815+0000","last_fullsized":"2023-10-06T01:40:42.484815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487398+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:40:42.487398+0000","last_peered":"2023-10-06T01:40:42.487398+0000","last_clean":"2023-10-06T01:40:42.487398+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:40:42.487398+0000","last_undegraded":"2023-10-06T01:40:42.487398+0000","last_fullsized":"2023-10-06T01:40:42.487398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481250+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:40:42.481250+0000","last_peered":"2023-10-06T01:40:42.481250+0000","last_clean":"2023-10-06T01:40:42.481250+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:40:42.481250+0000","last_undegraded":"2023-10-06T01:40:42.481250+0000","last_fullsized":"2023-10-06T01:40:42.481250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.492058+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:40:42.492058+0000","last_peered":"2023-10-06T01:40:42.492058+0000","last_clean":"2023-10-06T01:40:42.492058+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:40:42.492058+0000","last_undegraded":"2023-10-06T01:40:42.492058+0000","last_fullsized":"2023-10-06T01:40:42.492058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465919+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:42.465919+0000","last_peered":"2023-10-06T01:40:42.465919+0000","last_clean":"2023-10-06T01:40:42.465919+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:42.465919+0000","last_undegraded":"2023-10-06T01:40:42.465919+0000","last_fullsized":"2023-10-06T01:40:42.465919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490771+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:40:42.490771+0000","last_peered":"2023-10-06T01:40:42.490771+0000","last_clean":"2023-10-06T01:40:42.490771+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:40:42.490771+0000","last_undegraded":"2023-10-06T01:40:42.490771+0000","last_fullsized":"2023-10-06T01:40:42.490771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491052+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:40:42.491052+0000","last_peered":"2023-10-06T01:40:42.491052+0000","last_clean":"2023-10-06T01:40:42.491052+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:40:42.491052+0000","last_undegraded":"2023-10-06T01:40:42.491052+0000","last_fullsized":"2023-10-06T01:40:42.491052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486410+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:40:42.486410+0000","last_peered":"2023-10-06T01:40:42.486410+0000","last_clean":"2023-10-06T01:40:42.486410+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:40:42.486410+0000","last_undegraded":"2023-10-06T01:40:42.486410+0000","last_fullsized":"2023-10-06T01:40:42.486410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480949+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:40:42.480949+0000","last_peered":"2023-10-06T01:40:42.480949+0000","last_clean":"2023-10-06T01:40:42.480949+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:40:42.480949+0000","last_undegraded":"2023-10-06T01:40:42.480949+0000","last_fullsized":"2023-10-06T01:40:42.480949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484424+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:40:42.484424+0000","last_peered":"2023-10-06T01:40:42.484424+0000","last_clean":"2023-10-06T01:40:42.484424+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:40:42.484424+0000","last_undegraded":"2023-10-06T01:40:42.484424+0000","last_fullsized":"2023-10-06T01:40:42.484424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478903+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:40:42.478903+0000","last_peered":"2023-10-06T01:40:42.478903+0000","last_clean":"2023-10-06T01:40:42.478903+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:40:42.478903+0000","last_undegraded":"2023-10-06T01:40:42.478903+0000","last_fullsized":"2023-10-06T01:40:42.478903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486863+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:40:42.486863+0000","last_peered":"2023-10-06T01:40:42.486863+0000","last_clean":"2023-10-06T01:40:42.486863+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:40:42.486863+0000","last_undegraded":"2023-10-06T01:40:42.486863+0000","last_fullsized":"2023-10-06T01:40:42.486863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475400+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:42.475400+0000","last_peered":"2023-10-06T01:40:42.475400+0000","last_clean":"2023-10-06T01:40:42.475400+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:42.475400+0000","last_undegraded":"2023-10-06T01:40:42.475400+0000","last_fullsized":"2023-10-06T01:40:42.475400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486796+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:40:42.486796+0000","last_peered":"2023-10-06T01:40:42.486796+0000","last_clean":"2023-10-06T01:40:42.486796+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:40:42.486796+0000","last_undegraded":"2023-10-06T01:40:42.486796+0000","last_fullsized":"2023-10-06T01:40:42.486796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478912+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:42.478912+0000","last_peered":"2023-10-06T01:40:42.478912+0000","last_clean":"2023-10-06T01:40:42.478912+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:42.478912+0000","last_undegraded":"2023-10-06T01:40:42.478912+0000","last_fullsized":"2023-10-06T01:40:42.478912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.476094+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:40:42.476094+0000","last_peered":"2023-10-06T01:40:42.476094+0000","last_clean":"2023-10-06T01:40:42.476094+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:40:42.476094+0000","last_undegraded":"2023-10-06T01:40:42.476094+0000","last_fullsized":"2023-10-06T01:40:42.476094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.473067+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:42.473067+0000","last_peered":"2023-10-06T01:40:42.473067+0000","last_clean":"2023-10-06T01:40:42.473067+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:42.473067+0000","last_undegraded":"2023-10-06T01:40:42.473067+0000","last_fullsized":"2023-10-06T01:40:42.473067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.470737+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:41.470737+0000","last_peered":"2023-10-06T01:40:41.470737+0000","last_clean":"2023-10-06T01:40:41.470737+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:41.470737+0000","last_undegraded":"2023-10-06T01:40:41.470737+0000","last_fullsized":"2023-10-06T01:40:41.470737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491359+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:40:42.491359+0000","last_peered":"2023-10-06T01:40:42.491359+0000","last_clean":"2023-10-06T01:40:42.491359+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:40:42.491359+0000","last_undegraded":"2023-10-06T01:40:42.491359+0000","last_fullsized":"2023-10-06T01:40:42.491359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.462843+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:40:42.462843+0000","last_peered":"2023-10-06T01:40:42.462843+0000","last_clean":"2023-10-06T01:40:42.462843+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:40:42.462843+0000","last_undegraded":"2023-10-06T01:40:42.462843+0000","last_fullsized":"2023-10-06T01:40:42.462843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483351+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:42.483351+0000","last_peered":"2023-10-06T01:40:42.483351+0000","last_clean":"2023-10-06T01:40:42.483351+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:42.483351+0000","last_undegraded":"2023-10-06T01:40:42.483351+0000","last_fullsized":"2023-10-06T01:40:42.483351+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471793+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:41.471793+0000","last_peered":"2023-10-06T01:40:41.471793+0000","last_clean":"2023-10-06T01:40:41.471793+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:41.471793+0000","last_undegraded":"2023-10-06T01:40:41.471793+0000","last_fullsized":"2023-10-06T01:40:41.471793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484557+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:40:42.484557+0000","last_peered":"2023-10-06T01:40:42.484557+0000","last_clean":"2023-10-06T01:40:42.484557+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:40:42.484557+0000","last_undegraded":"2023-10-06T01:40:42.484557+0000","last_fullsized":"2023-10-06T01:40:42.484557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490915+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:40:42.490915+0000","last_peered":"2023-10-06T01:40:42.490915+0000","last_clean":"2023-10-06T01:40:42.490915+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:40:42.490915+0000","last_undegraded":"2023-10-06T01:40:42.490915+0000","last_fullsized":"2023-10-06T01:40:42.490915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478379+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:42.478379+0000","last_peered":"2023-10-06T01:40:42.478379+0000","last_clean":"2023-10-06T01:40:42.478379+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:42.478379+0000","last_undegraded":"2023-10-06T01:40:42.478379+0000","last_fullsized":"2023-10-06T01:40:42.478379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.493970+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:40:42.493970+0000","last_peered":"2023-10-06T01:40:42.493970+0000","last_clean":"2023-10-06T01:40:42.493970+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:40:42.493970+0000","last_undegraded":"2023-10-06T01:40:42.493970+0000","last_fullsized":"2023-10-06T01:40:42.493970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484065+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:42.484065+0000","last_peered":"2023-10-06T01:40:42.484065+0000","last_clean":"2023-10-06T01:40:42.484065+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:42.484065+0000","last_undegraded":"2023-10-06T01:40:42.484065+0000","last_fullsized":"2023-10-06T01:40:42.484065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488369+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:40:42.488369+0000","last_peered":"2023-10-06T01:40:42.488369+0000","last_clean":"2023-10-06T01:40:42.488369+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:40:42.488369+0000","last_undegraded":"2023-10-06T01:40:42.488369+0000","last_fullsized":"2023-10-06T01:40:42.488369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478175+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:42.478175+0000","last_peered":"2023-10-06T01:40:42.478175+0000","last_clean":"2023-10-06T01:40:42.478175+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:42.478175+0000","last_undegraded":"2023-10-06T01:40:42.478175+0000","last_fullsized":"2023-10-06T01:40:42.478175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469395+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:41.469395+0000","last_peered":"2023-10-06T01:40:41.469395+0000","last_clean":"2023-10-06T01:40:41.469395+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:41.469395+0000","last_undegraded":"2023-10-06T01:40:41.469395+0000","last_fullsized":"2023-10-06T01:40:41.469395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484499+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:40:42.484499+0000","last_peered":"2023-10-06T01:40:42.484499+0000","last_clean":"2023-10-06T01:40:42.484499+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:40:42.484499+0000","last_undegraded":"2023-10-06T01:40:42.484499+0000","last_fullsized":"2023-10-06T01:40:42.484499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466077+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:40:42.466077+0000","last_peered":"2023-10-06T01:40:42.466077+0000","last_clean":"2023-10-06T01:40:42.466077+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:40:42.466077+0000","last_undegraded":"2023-10-06T01:40:42.466077+0000","last_fullsized":"2023-10-06T01:40:42.466077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481908+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:42.481908+0000","last_peered":"2023-10-06T01:40:42.481908+0000","last_clean":"2023-10-06T01:40:42.481908+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:42.481908+0000","last_undegraded":"2023-10-06T01:40:42.481908+0000","last_fullsized":"2023-10-06T01:40:42.481908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491752+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:40:42.491752+0000","last_peered":"2023-10-06T01:40:42.491752+0000","last_clean":"2023-10-06T01:40:42.491752+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:40:42.491752+0000","last_undegraded":"2023-10-06T01:40:42.491752+0000","last_fullsized":"2023-10-06T01:40:42.491752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488194+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:40:42.488194+0000","last_peered":"2023-10-06T01:40:42.488194+0000","last_clean":"2023-10-06T01:40:42.488194+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:40:42.488194+0000","last_undegraded":"2023-10-06T01:40:42.488194+0000","last_fullsized":"2023-10-06T01:40:42.488194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487367+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:40:42.487367+0000","last_peered":"2023-10-06T01:40:42.487367+0000","last_clean":"2023-10-06T01:40:42.487367+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:40:42.487367+0000","last_undegraded":"2023-10-06T01:40:42.487367+0000","last_fullsized":"2023-10-06T01:40:42.487367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483508+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:42.483508+0000","last_peered":"2023-10-06T01:40:42.483508+0000","last_clean":"2023-10-06T01:40:42.483508+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:42.483508+0000","last_undegraded":"2023-10-06T01:40:42.483508+0000","last_fullsized":"2023-10-06T01:40:42.483508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483166+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:40:42.483166+0000","last_peered":"2023-10-06T01:40:42.483166+0000","last_clean":"2023-10-06T01:40:42.483166+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:40:42.483166+0000","last_undegraded":"2023-10-06T01:40:42.483166+0000","last_fullsized":"2023-10-06T01:40:42.483166+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.474713+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:42.474713+0000","last_peered":"2023-10-06T01:40:42.474713+0000","last_clean":"2023-10-06T01:40:42.474713+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:42.474713+0000","last_undegraded":"2023-10-06T01:40:42.474713+0000","last_fullsized":"2023-10-06T01:40:42.474713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486018+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:40:42.486018+0000","last_peered":"2023-10-06T01:40:42.486018+0000","last_clean":"2023-10-06T01:40:42.486018+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:40:42.486018+0000","last_undegraded":"2023-10-06T01:40:42.486018+0000","last_fullsized":"2023-10-06T01:40:42.486018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.471507+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:42.471507+0000","last_peered":"2023-10-06T01:40:42.471507+0000","last_clean":"2023-10-06T01:40:42.471507+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:42.471507+0000","last_undegraded":"2023-10-06T01:40:42.471507+0000","last_fullsized":"2023-10-06T01:40:42.471507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466570+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:40:42.466570+0000","last_peered":"2023-10-06T01:40:42.466570+0000","last_clean":"2023-10-06T01:40:42.466570+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:40:42.466570+0000","last_undegraded":"2023-10-06T01:40:42.466570+0000","last_fullsized":"2023-10-06T01:40:42.466570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484527+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:40:42.484527+0000","last_peered":"2023-10-06T01:40:42.484527+0000","last_clean":"2023-10-06T01:40:42.484527+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:40:42.484527+0000","last_undegraded":"2023-10-06T01:40:42.484527+0000","last_fullsized":"2023-10-06T01:40:42.484527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488076+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:40:42.488076+0000","last_peered":"2023-10-06T01:40:42.488076+0000","last_clean":"2023-10-06T01:40:42.488076+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:40:42.488076+0000","last_undegraded":"2023-10-06T01:40:42.488076+0000","last_fullsized":"2023-10-06T01:40:42.488076+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481475+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:40:42.481475+0000","last_peered":"2023-10-06T01:40:42.481475+0000","last_clean":"2023-10-06T01:40:42.481475+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:40:42.481475+0000","last_undegraded":"2023-10-06T01:40:42.481475+0000","last_fullsized":"2023-10-06T01:40:42.481475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475995+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:40:42.475995+0000","last_peered":"2023-10-06T01:40:42.475995+0000","last_clean":"2023-10-06T01:40:42.475995+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:40:42.475995+0000","last_undegraded":"2023-10-06T01:40:42.475995+0000","last_fullsized":"2023-10-06T01:40:42.475995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477776+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:40:42.477776+0000","last_peered":"2023-10-06T01:40:42.477776+0000","last_clean":"2023-10-06T01:40:42.477776+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:40:42.477776+0000","last_undegraded":"2023-10-06T01:40:42.477776+0000","last_fullsized":"2023-10-06T01:40:42.477776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.474303+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:41.474303+0000","last_peered":"2023-10-06T01:40:41.474303+0000","last_clean":"2023-10-06T01:40:41.474303+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:41.474303+0000","last_undegraded":"2023-10-06T01:40:41.474303+0000","last_fullsized":"2023-10-06T01:40:41.474303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478522+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:42.478522+0000","last_peered":"2023-10-06T01:40:42.478522+0000","last_clean":"2023-10-06T01:40:42.478522+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:42.478522+0000","last_undegraded":"2023-10-06T01:40:42.478522+0000","last_fullsized":"2023-10-06T01:40:42.478522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469403+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:41.469403+0000","last_peered":"2023-10-06T01:40:41.469403+0000","last_clean":"2023-10-06T01:40:41.469403+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:41.469403+0000","last_undegraded":"2023-10-06T01:40:41.469403+0000","last_fullsized":"2023-10-06T01:40:41.469403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484765+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:40:42.484765+0000","last_peered":"2023-10-06T01:40:42.484765+0000","last_clean":"2023-10-06T01:40:42.484765+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:40:42.484765+0000","last_undegraded":"2023-10-06T01:40:42.484765+0000","last_fullsized":"2023-10-06T01:40:42.484765+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.462312+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:40:42.462312+0000","last_peered":"2023-10-06T01:40:42.462312+0000","last_clean":"2023-10-06T01:40:42.462312+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:40:42.462312+0000","last_undegraded":"2023-10-06T01:40:42.462312+0000","last_fullsized":"2023-10-06T01:40:42.462312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.463248+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:40:42.463248+0000","last_peered":"2023-10-06T01:40:42.463248+0000","last_clean":"2023-10-06T01:40:42.463248+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:40:42.463248+0000","last_undegraded":"2023-10-06T01:40:42.463248+0000","last_fullsized":"2023-10-06T01:40:42.463248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484843+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:40:42.484843+0000","last_peered":"2023-10-06T01:40:42.484843+0000","last_clean":"2023-10-06T01:40:42.484843+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:40:42.484843+0000","last_undegraded":"2023-10-06T01:40:42.484843+0000","last_fullsized":"2023-10-06T01:40:42.484843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477701+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:41.477701+0000","last_peered":"2023-10-06T01:40:41.477701+0000","last_clean":"2023-10-06T01:40:41.477701+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:41.477701+0000","last_undegraded":"2023-10-06T01:40:41.477701+0000","last_fullsized":"2023-10-06T01:40:41.477701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466644+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:40:42.466644+0000","last_peered":"2023-10-06T01:40:42.466644+0000","last_clean":"2023-10-06T01:40:42.466644+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:40:42.466644+0000","last_undegraded":"2023-10-06T01:40:42.466644+0000","last_fullsized":"2023-10-06T01:40:42.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475832+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:42.475832+0000","last_peered":"2023-10-06T01:40:42.475832+0000","last_clean":"2023-10-06T01:40:42.475832+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:42.475832+0000","last_undegraded":"2023-10-06T01:40:42.475832+0000","last_fullsized":"2023-10-06T01:40:42.475832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460317+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:40:42.460317+0000","last_peered":"2023-10-06T01:40:42.460317+0000","last_clean":"2023-10-06T01:40:42.460317+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:40:42.460317+0000","last_undegraded":"2023-10-06T01:40:42.460317+0000","last_fullsized":"2023-10-06T01:40:42.460317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460448+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:40:42.460448+0000","last_peered":"2023-10-06T01:40:42.460448+0000","last_clean":"2023-10-06T01:40:42.460448+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:40:42.460448+0000","last_undegraded":"2023-10-06T01:40:42.460448+0000","last_fullsized":"2023-10-06T01:40:42.460448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456861+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:41.456861+0000","last_peered":"2023-10-06T01:40:41.456861+0000","last_clean":"2023-10-06T01:40:41.456861+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:41.456861+0000","last_undegraded":"2023-10-06T01:40:41.456861+0000","last_fullsized":"2023-10-06T01:40:41.456861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.479190+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:42.479190+0000","last_peered":"2023-10-06T01:40:42.479190+0000","last_clean":"2023-10-06T01:40:42.479190+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:42.479190+0000","last_undegraded":"2023-10-06T01:40:42.479190+0000","last_fullsized":"2023-10-06T01:40:42.479190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.477681+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:41.477681+0000","last_peered":"2023-10-06T01:40:41.477681+0000","last_clean":"2023-10-06T01:40:41.477681+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:41.477681+0000","last_undegraded":"2023-10-06T01:40:41.477681+0000","last_fullsized":"2023-10-06T01:40:41.477681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.458496+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:41.458496+0000","last_peered":"2023-10-06T01:40:41.458496+0000","last_clean":"2023-10-06T01:40:41.458496+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:41.458496+0000","last_undegraded":"2023-10-06T01:40:41.458496+0000","last_fullsized":"2023-10-06T01:40:41.458496+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480021+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:42.480021+0000","last_peered":"2023-10-06T01:40:42.480021+0000","last_clean":"2023-10-06T01:40:42.480021+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:42.480021+0000","last_undegraded":"2023-10-06T01:40:42.480021+0000","last_fullsized":"2023-10-06T01:40:42.480021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485178+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:40:42.485178+0000","last_peered":"2023-10-06T01:40:42.485178+0000","last_clean":"2023-10-06T01:40:42.485178+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:40:42.485178+0000","last_undegraded":"2023-10-06T01:40:42.485178+0000","last_fullsized":"2023-10-06T01:40:42.485178+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.456774+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:41.456774+0000","last_peered":"2023-10-06T01:40:41.456774+0000","last_clean":"2023-10-06T01:40:41.456774+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:41.456774+0000","last_undegraded":"2023-10-06T01:40:41.456774+0000","last_fullsized":"2023-10-06T01:40:41.456774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485111+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:42.485111+0000","last_peered":"2023-10-06T01:40:42.485111+0000","last_clean":"2023-10-06T01:40:42.485111+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:42.485111+0000","last_undegraded":"2023-10-06T01:40:42.485111+0000","last_fullsized":"2023-10-06T01:40:42.485111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-06T01:40:43.565503+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:40:43.565503+0000","last_peered":"2023-10-06T01:40:43.565503+0000","last_clean":"2023-10-06T01:40:43.565503+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:40:43.565503+0000","last_undegraded":"2023-10-06T01:40:43.565503+0000","last_fullsized":"2023-10-06T01:40:43.565503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.476205+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:40:42.476205+0000","last_peered":"2023-10-06T01:40:42.476205+0000","last_clean":"2023-10-06T01:40:42.476205+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:40:42.476205+0000","last_undegraded":"2023-10-06T01:40:42.476205+0000","last_fullsized":"2023-10-06T01:40:42.476205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484862+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:40:42.484862+0000","last_peered":"2023-10-06T01:40:42.484862+0000","last_clean":"2023-10-06T01:40:42.484862+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:40:42.484862+0000","last_undegraded":"2023-10-06T01:40:42.484862+0000","last_fullsized":"2023-10-06T01:40:42.484862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460308+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:40:42.460308+0000","last_peered":"2023-10-06T01:40:42.460308+0000","last_clean":"2023-10-06T01:40:42.460308+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:40:42.460308+0000","last_undegraded":"2023-10-06T01:40:42.460308+0000","last_fullsized":"2023-10-06T01:40:42.460308+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469064+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:41.469064+0000","last_peered":"2023-10-06T01:40:41.469064+0000","last_clean":"2023-10-06T01:40:41.469064+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:41.469064+0000","last_undegraded":"2023-10-06T01:40:41.469064+0000","last_fullsized":"2023-10-06T01:40:41.469064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478667+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:40:42.478667+0000","last_peered":"2023-10-06T01:40:42.478667+0000","last_clean":"2023-10-06T01:40:42.478667+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:40:42.478667+0000","last_undegraded":"2023-10-06T01:40:42.478667+0000","last_fullsized":"2023-10-06T01:40:42.478667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.471786+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:41.471786+0000","last_peered":"2023-10-06T01:40:41.471786+0000","last_clean":"2023-10-06T01:40:41.471786+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:41.471786+0000","last_undegraded":"2023-10-06T01:40:41.471786+0000","last_fullsized":"2023-10-06T01:40:41.471786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T01:40:41.469790+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:41.469790+0000","last_peered":"2023-10-06T01:40:41.469790+0000","last_clean":"2023-10-06T01:40:41.469790+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:41.469790+0000","last_undegraded":"2023-10-06T01:40:41.469790+0000","last_fullsized":"2023-10-06T01:40:41.469790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":766304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297688,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074152,"statfs":{"total":96636764160,"available":96331931648,"internally_reserved":0,"allocated":409600,"data_stored":176490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":1011712,"data_stored":773245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25700000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.23300000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.5},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":175880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":183012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.376},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.438}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.504}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":183417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.307}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.503}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.437},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":184343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":417792,"data_stored":182877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.372},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.5}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41499999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:50.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:40:50.380 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:40:51.027 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:40:51.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.153+0000 7f2ac3fc8700 1 Processor -- start 2023-10-06T01:40:51.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.153+0000 7f2ac3fc8700 1 -- start start 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac3fc8700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2abc0b07a0 0x7f2abc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac3fc8700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0b0ca0 con 0x7f2abc0b49e0 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac3fc8700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0b0f80 con 0x7f2abc1595a0 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac3fc8700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc0ab390 con 0x7f2abc0b07a0 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac2565700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2abc0b07a0 0x7f2abc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac1563700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2abc0b49e0 0x7f2abc0aac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45672/0 (socket says 172.21.15.98:45672) 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac1563700 1 -- 172.21.15.98:0/20157465 learned_addr learned my addr 172.21.15.98:0/20157465 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:51.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac0d62700 1 -- 172.21.15.98:0/20157465 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613483851 0 0) 0x7f2abc0b0ca0 con 0x7f2abc0b49e0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac0d62700 1 -- 172.21.15.98:0/20157465 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa8003570 con 0x7f2abc0b49e0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac2565700 1 -- 172.21.15.98:0/20157465 >> v1:172.21.15.184:6789/0 conn(0x7f2abc1595a0 legacy=0x7f2abc0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac2565700 1 -- 172.21.15.98:0/20157465 >> v1:172.21.15.98:6789/0 conn(0x7f2abc0b49e0 legacy=0x7f2abc0aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.154+0000 7f2ac2565700 1 -- 172.21.15.98:0/20157465 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0b1260 con 0x7f2abc0b07a0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac2565700 1 --2- 172.21.15.98:0/20157465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc0b07a0 0x7f2abc0ccfd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009e90 tx=0x7f2ab4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529721cfb5fe9171 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac0d62700 1 -- 172.21.15.98:0/20157465 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400e070 con 0x7f2abc0b07a0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac0d62700 1 -- 172.21.15.98:0/20157465 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab400bc40 con 0x7f2abc0b07a0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac0d62700 1 -- 172.21.15.98:0/20157465 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4012430 con 0x7f2abc0b07a0 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/20157465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc0b07a0 msgr2=0x7f2abc0ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/20157465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc0b07a0 0x7f2abc0ccfd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009e90 tx=0x7f2ab4009780 comp rx=0 tx=0).stop 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/20157465 shutdown_connections 2023-10-06T01:40:51.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/20157465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc0b07a0 0x7f2abc0ccfd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/20157465 >> 172.21.15.98:0/20157465 conn(0x7f2abc099830 msgr2=0x7f2abc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:51.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/20157465 shutdown_connections 2023-10-06T01:40:51.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.155+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/20157465 wait complete. 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 Processor -- start 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 -- start start 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2abc0b07a0 0x7f2abc15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 0x7f2abc16a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc169ac0 con 0x7f2abc1595a0 2023-10-06T01:40:51.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc16a040 con 0x7f2abc0b49e0 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac3fc8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc15d070 con 0x7f2abc0b07a0 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac2565700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac2565700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47040/0 (socket says 172.21.15.98:47040) 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.157+0000 7f2ac2565700 1 -- 172.21.15.98:0/2748579104 learned_addr learned my addr 172.21.15.98:0/2748579104 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac1563700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 0x7f2abc16a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac1d64700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2abc0b07a0 0x7f2abc15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 -- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2abc0b07a0 msgr2=0x7f2abc15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2abc0b07a0 0x7f2abc15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 -- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 msgr2=0x7f2abc16a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 0x7f2abc16a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 -- 172.21.15.98:0/2748579104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009a20 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac1563700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 0x7f2abc16a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac2565700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2abc067c60 tx=0x7f2ab4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab401b630 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab401b7d0 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab401bb90 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc16db10 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.158+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc169cd0 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.159+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc16e030 con 0x7f2abc1595a0 2023-10-06T01:40:51.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.159+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ab401c400 con 0x7f2abc1595a0 2023-10-06T01:40:51.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.160+0000 7f2ab1ffb700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 0x7f2aa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.160+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2ab40865e0 con 0x7f2abc1595a0 2023-10-06T01:40:51.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.160+0000 7f2ac1d64700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 0x7f2aa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.160+0000 7f2ac1d64700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 0x7f2aa00693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0af5e0 tx=0x7f2ab8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.164+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2abc16e030 con 0x7f2abc1595a0 2023-10-06T01:40:51.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.343+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2abc16e030 con 0x7f2aa0066f30 2023-10-06T01:40:51.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.351+0000 7f2ab1ffb700 1 -- 172.21.15.98:0/2748579104 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311638 (secure 0 0 0) 0x7f2abc16e030 con 0x7f2aa0066f30 2023-10-06T01:40:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:51.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 msgr2=0x7f2aa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 0x7f2aa00693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0af5e0 tx=0x7f2ab8009040 comp rx=0 tx=0).stop 2023-10-06T01:40:51.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 msgr2=0x7f2abc16d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2abc067c60 tx=0x7f2ab4019000 comp rx=0 tx=0).stop 2023-10-06T01:40:51.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 shutdown_connections 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2aa0066f30 0x7f2aa00693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2abc1595a0 0x7f2abc16d0c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2abc0b07a0 0x7f2abc15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 --2- 172.21.15.98:0/2748579104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2abc0b49e0 0x7f2abc16a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 >> 172.21.15.98:0/2748579104 conn(0x7f2abc099830 msgr2=0x7f2abc158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 shutdown_connections 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.355+0000 7f2ac3fc8700 1 -- 172.21.15.98:0/2748579104 wait complete. 2023-10-06T01:40:51.361 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:40:51.367 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-06T01:40:49.798272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382756,"kb_used_data":4452,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591964,"statfs":{"total":773094113280,"available":770654171136,"internally_reserved":0,"allocated":4558848,"data_stored":2646451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001396"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486569+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:42.486569+0000","last_peered":"2023-10-06T01:40:42.486569+0000","last_clean":"2023-10-06T01:40:42.486569+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:42.486569+0000","last_undegraded":"2023-10-06T01:40:42.486569+0000","last_fullsized":"2023-10-06T01:40:42.486569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491145+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:40:42.491145+0000","last_peered":"2023-10-06T01:40:42.491145+0000","last_clean":"2023-10-06T01:40:42.491145+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:40:42.491145+0000","last_undegraded":"2023-10-06T01:40:42.491145+0000","last_fullsized":"2023-10-06T01:40:42.491145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478857+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:40:42.478857+0000","last_peered":"2023-10-06T01:40:42.478857+0000","last_clean":"2023-10-06T01:40:42.478857+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:40:42.478857+0000","last_undegraded":"2023-10-06T01:40:42.478857+0000","last_fullsized":"2023-10-06T01:40:42.478857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488057+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:42.488057+0000","last_peered":"2023-10-06T01:40:42.488057+0000","last_clean":"2023-10-06T01:40:42.488057+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:42.488057+0000","last_undegraded":"2023-10-06T01:40:42.488057+0000","last_fullsized":"2023-10-06T01:40:42.488057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491125+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:40:42.491125+0000","last_peered":"2023-10-06T01:40:42.491125+0000","last_clean":"2023-10-06T01:40:42.491125+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:40:42.491125+0000","last_undegraded":"2023-10-06T01:40:42.491125+0000","last_fullsized":"2023-10-06T01:40:42.491125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484825+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:40:42.484825+0000","last_peered":"2023-10-06T01:40:42.484825+0000","last_clean":"2023-10-06T01:40:42.484825+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:40:42.484825+0000","last_undegraded":"2023-10-06T01:40:42.484825+0000","last_fullsized":"2023-10-06T01:40:42.484825+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485815+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:42.485815+0000","last_peered":"2023-10-06T01:40:42.485815+0000","last_clean":"2023-10-06T01:40:42.485815+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:42.485815+0000","last_undegraded":"2023-10-06T01:40:42.485815+0000","last_fullsized":"2023-10-06T01:40:42.485815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488266+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:42.488266+0000","last_peered":"2023-10-06T01:40:42.488266+0000","last_clean":"2023-10-06T01:40:42.488266+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:42.488266+0000","last_undegraded":"2023-10-06T01:40:42.488266+0000","last_fullsized":"2023-10-06T01:40:42.488266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487227+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:42.487227+0000","last_peered":"2023-10-06T01:40:42.487227+0000","last_clean":"2023-10-06T01:40:42.487227+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:42.487227+0000","last_undegraded":"2023-10-06T01:40:42.487227+0000","last_fullsized":"2023-10-06T01:40:42.487227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487976+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:42.487976+0000","last_peered":"2023-10-06T01:40:42.487976+0000","last_clean":"2023-10-06T01:40:42.487976+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:42.487976+0000","last_undegraded":"2023-10-06T01:40:42.487976+0000","last_fullsized":"2023-10-06T01:40:42.487976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488051+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:42.488051+0000","last_peered":"2023-10-06T01:40:42.488051+0000","last_clean":"2023-10-06T01:40:42.488051+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:42.488051+0000","last_undegraded":"2023-10-06T01:40:42.488051+0000","last_fullsized":"2023-10-06T01:40:42.488051+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491590+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:40:42.491590+0000","last_peered":"2023-10-06T01:40:42.491590+0000","last_clean":"2023-10-06T01:40:42.491590+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:40:42.491590+0000","last_undegraded":"2023-10-06T01:40:42.491590+0000","last_fullsized":"2023-10-06T01:40:42.491590+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482369+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:40:42.482369+0000","last_peered":"2023-10-06T01:40:42.482369+0000","last_clean":"2023-10-06T01:40:42.482369+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:40:42.482369+0000","last_undegraded":"2023-10-06T01:40:42.482369+0000","last_fullsized":"2023-10-06T01:40:42.482369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488082+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:42.488082+0000","last_peered":"2023-10-06T01:40:42.488082+0000","last_clean":"2023-10-06T01:40:42.488082+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:42.488082+0000","last_undegraded":"2023-10-06T01:40:42.488082+0000","last_fullsized":"2023-10-06T01:40:42.488082+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484796+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:40:42.484796+0000","last_peered":"2023-10-06T01:40:42.484796+0000","last_clean":"2023-10-06T01:40:42.484796+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:40:42.484796+0000","last_undegraded":"2023-10-06T01:40:42.484796+0000","last_fullsized":"2023-10-06T01:40:42.484796+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490712+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:42.490712+0000","last_peered":"2023-10-06T01:40:42.490712+0000","last_clean":"2023-10-06T01:40:42.490712+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:42.490712+0000","last_undegraded":"2023-10-06T01:40:42.490712+0000","last_fullsized":"2023-10-06T01:40:42.490712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490826+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:40:42.490826+0000","last_peered":"2023-10-06T01:40:42.490826+0000","last_clean":"2023-10-06T01:40:42.490826+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:40:42.490826+0000","last_undegraded":"2023-10-06T01:40:42.490826+0000","last_fullsized":"2023-10-06T01:40:42.490826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487013+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:40:42.487013+0000","last_peered":"2023-10-06T01:40:42.487013+0000","last_clean":"2023-10-06T01:40:42.487013+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:40:42.487013+0000","last_undegraded":"2023-10-06T01:40:42.487013+0000","last_fullsized":"2023-10-06T01:40:42.487013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491549+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:42.491549+0000","last_peered":"2023-10-06T01:40:42.491549+0000","last_clean":"2023-10-06T01:40:42.491549+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:42.491549+0000","last_undegraded":"2023-10-06T01:40:42.491549+0000","last_fullsized":"2023-10-06T01:40:42.491549+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488301+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:40:42.488301+0000","last_peered":"2023-10-06T01:40:42.488301+0000","last_clean":"2023-10-06T01:40:42.488301+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:40:42.488301+0000","last_undegraded":"2023-10-06T01:40:42.488301+0000","last_fullsized":"2023-10-06T01:40:42.488301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484798+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:40:42.484798+0000","last_peered":"2023-10-06T01:40:42.484798+0000","last_clean":"2023-10-06T01:40:42.484798+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:40:42.484798+0000","last_undegraded":"2023-10-06T01:40:42.484798+0000","last_fullsized":"2023-10-06T01:40:42.484798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488100+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:42.488100+0000","last_peered":"2023-10-06T01:40:42.488100+0000","last_clean":"2023-10-06T01:40:42.488100+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:42.488100+0000","last_undegraded":"2023-10-06T01:40:42.488100+0000","last_fullsized":"2023-10-06T01:40:42.488100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481933+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:42.481933+0000","last_peered":"2023-10-06T01:40:42.481933+0000","last_clean":"2023-10-06T01:40:42.481933+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:42.481933+0000","last_undegraded":"2023-10-06T01:40:42.481933+0000","last_fullsized":"2023-10-06T01:40:42.481933+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482763+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:42.482763+0000","last_peered":"2023-10-06T01:40:42.482763+0000","last_clean":"2023-10-06T01:40:42.482763+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:42.482763+0000","last_undegraded":"2023-10-06T01:40:42.482763+0000","last_fullsized":"2023-10-06T01:40:42.482763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485630+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:42.485630+0000","last_peered":"2023-10-06T01:40:42.485630+0000","last_clean":"2023-10-06T01:40:42.485630+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:42.485630+0000","last_undegraded":"2023-10-06T01:40:42.485630+0000","last_fullsized":"2023-10-06T01:40:42.485630+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481527+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:40:42.481527+0000","last_peered":"2023-10-06T01:40:42.481527+0000","last_clean":"2023-10-06T01:40:42.481527+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:40:42.481527+0000","last_undegraded":"2023-10-06T01:40:42.481527+0000","last_fullsized":"2023-10-06T01:40:42.481527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490786+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:40:42.490786+0000","last_peered":"2023-10-06T01:40:42.490786+0000","last_clean":"2023-10-06T01:40:42.490786+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:40:42.490786+0000","last_undegraded":"2023-10-06T01:40:42.490786+0000","last_fullsized":"2023-10-06T01:40:42.490786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491609+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:40:42.491609+0000","last_peered":"2023-10-06T01:40:42.491609+0000","last_clean":"2023-10-06T01:40:42.491609+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:40:42.491609+0000","last_undegraded":"2023-10-06T01:40:42.491609+0000","last_fullsized":"2023-10-06T01:40:42.491609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490628+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:42.490628+0000","last_peered":"2023-10-06T01:40:42.490628+0000","last_clean":"2023-10-06T01:40:42.490628+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:42.490628+0000","last_undegraded":"2023-10-06T01:40:42.490628+0000","last_fullsized":"2023-10-06T01:40:42.490628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478891+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:40:42.478891+0000","last_peered":"2023-10-06T01:40:42.478891+0000","last_clean":"2023-10-06T01:40:42.478891+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:40:42.478891+0000","last_undegraded":"2023-10-06T01:40:42.478891+0000","last_fullsized":"2023-10-06T01:40:42.478891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484815+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:40:42.484815+0000","last_peered":"2023-10-06T01:40:42.484815+0000","last_clean":"2023-10-06T01:40:42.484815+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:40:42.484815+0000","last_undegraded":"2023-10-06T01:40:42.484815+0000","last_fullsized":"2023-10-06T01:40:42.484815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487398+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:40:42.487398+0000","last_peered":"2023-10-06T01:40:42.487398+0000","last_clean":"2023-10-06T01:40:42.487398+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:40:42.487398+0000","last_undegraded":"2023-10-06T01:40:42.487398+0000","last_fullsized":"2023-10-06T01:40:42.487398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481250+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:40:42.481250+0000","last_peered":"2023-10-06T01:40:42.481250+0000","last_clean":"2023-10-06T01:40:42.481250+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:40:42.481250+0000","last_undegraded":"2023-10-06T01:40:42.481250+0000","last_fullsized":"2023-10-06T01:40:42.481250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.492058+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:40:42.492058+0000","last_peered":"2023-10-06T01:40:42.492058+0000","last_clean":"2023-10-06T01:40:42.492058+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:40:42.492058+0000","last_undegraded":"2023-10-06T01:40:42.492058+0000","last_fullsized":"2023-10-06T01:40:42.492058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477405+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:42.477405+0000","last_peered":"2023-10-06T01:40:42.477405+0000","last_clean":"2023-10-06T01:40:42.477405+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:42.477405+0000","last_undegraded":"2023-10-06T01:40:42.477405+0000","last_fullsized":"2023-10-06T01:40:42.477405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490771+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:40:42.490771+0000","last_peered":"2023-10-06T01:40:42.490771+0000","last_clean":"2023-10-06T01:40:42.490771+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:40:42.490771+0000","last_undegraded":"2023-10-06T01:40:42.490771+0000","last_fullsized":"2023-10-06T01:40:42.490771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491052+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:40:42.491052+0000","last_peered":"2023-10-06T01:40:42.491052+0000","last_clean":"2023-10-06T01:40:42.491052+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:40:42.491052+0000","last_undegraded":"2023-10-06T01:40:42.491052+0000","last_fullsized":"2023-10-06T01:40:42.491052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486410+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:40:42.486410+0000","last_peered":"2023-10-06T01:40:42.486410+0000","last_clean":"2023-10-06T01:40:42.486410+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:40:42.486410+0000","last_undegraded":"2023-10-06T01:40:42.486410+0000","last_fullsized":"2023-10-06T01:40:42.486410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.480949+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:40:42.480949+0000","last_peered":"2023-10-06T01:40:42.480949+0000","last_clean":"2023-10-06T01:40:42.480949+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:40:42.480949+0000","last_undegraded":"2023-10-06T01:40:42.480949+0000","last_fullsized":"2023-10-06T01:40:42.480949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486614+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:40:42.486614+0000","last_peered":"2023-10-06T01:40:42.486614+0000","last_clean":"2023-10-06T01:40:42.486614+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:40:42.486614+0000","last_undegraded":"2023-10-06T01:40:42.486614+0000","last_fullsized":"2023-10-06T01:40:42.486614+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478903+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:40:42.478903+0000","last_peered":"2023-10-06T01:40:42.478903+0000","last_clean":"2023-10-06T01:40:42.478903+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:40:42.478903+0000","last_undegraded":"2023-10-06T01:40:42.478903+0000","last_fullsized":"2023-10-06T01:40:42.478903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486863+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:40:42.486863+0000","last_peered":"2023-10-06T01:40:42.486863+0000","last_clean":"2023-10-06T01:40:42.486863+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:40:42.486863+0000","last_undegraded":"2023-10-06T01:40:42.486863+0000","last_fullsized":"2023-10-06T01:40:42.486863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483663+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:42.483663+0000","last_peered":"2023-10-06T01:40:42.483663+0000","last_clean":"2023-10-06T01:40:42.483663+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:42.483663+0000","last_undegraded":"2023-10-06T01:40:42.483663+0000","last_fullsized":"2023-10-06T01:40:42.483663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486796+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:40:42.486796+0000","last_peered":"2023-10-06T01:40:42.486796+0000","last_clean":"2023-10-06T01:40:42.486796+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:40:42.486796+0000","last_undegraded":"2023-10-06T01:40:42.486796+0000","last_fullsized":"2023-10-06T01:40:42.486796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488244+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:42.488244+0000","last_peered":"2023-10-06T01:40:42.488244+0000","last_clean":"2023-10-06T01:40:42.488244+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:42.488244+0000","last_undegraded":"2023-10-06T01:40:42.488244+0000","last_fullsized":"2023-10-06T01:40:42.488244+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.476094+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:40:42.476094+0000","last_peered":"2023-10-06T01:40:42.476094+0000","last_clean":"2023-10-06T01:40:42.476094+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:40:42.476094+0000","last_undegraded":"2023-10-06T01:40:42.476094+0000","last_fullsized":"2023-10-06T01:40:42.476094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481956+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:42.481956+0000","last_peered":"2023-10-06T01:40:42.481956+0000","last_clean":"2023-10-06T01:40:42.481956+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:42.481956+0000","last_undegraded":"2023-10-06T01:40:42.481956+0000","last_fullsized":"2023-10-06T01:40:42.481956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465481+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:42.465481+0000","last_peered":"2023-10-06T01:40:42.465481+0000","last_clean":"2023-10-06T01:40:42.465481+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:42.465481+0000","last_undegraded":"2023-10-06T01:40:42.465481+0000","last_fullsized":"2023-10-06T01:40:42.465481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491359+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:40:42.491359+0000","last_peered":"2023-10-06T01:40:42.491359+0000","last_clean":"2023-10-06T01:40:42.491359+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:40:42.491359+0000","last_undegraded":"2023-10-06T01:40:42.491359+0000","last_fullsized":"2023-10-06T01:40:42.491359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.462843+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:40:42.462843+0000","last_peered":"2023-10-06T01:40:42.462843+0000","last_clean":"2023-10-06T01:40:42.462843+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:40:42.462843+0000","last_undegraded":"2023-10-06T01:40:42.462843+0000","last_fullsized":"2023-10-06T01:40:42.462843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487911+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:42.487911+0000","last_peered":"2023-10-06T01:40:42.487911+0000","last_clean":"2023-10-06T01:40:42.487911+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:42.487911+0000","last_undegraded":"2023-10-06T01:40:42.487911+0000","last_fullsized":"2023-10-06T01:40:42.487911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.467196+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:42.467196+0000","last_peered":"2023-10-06T01:40:42.467196+0000","last_clean":"2023-10-06T01:40:42.467196+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:42.467196+0000","last_undegraded":"2023-10-06T01:40:42.467196+0000","last_fullsized":"2023-10-06T01:40:42.467196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484557+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:40:42.484557+0000","last_peered":"2023-10-06T01:40:42.484557+0000","last_clean":"2023-10-06T01:40:42.484557+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:40:42.484557+0000","last_undegraded":"2023-10-06T01:40:42.484557+0000","last_fullsized":"2023-10-06T01:40:42.484557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491736+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:40:42.491736+0000","last_peered":"2023-10-06T01:40:42.491736+0000","last_clean":"2023-10-06T01:40:42.491736+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:40:42.491736+0000","last_undegraded":"2023-10-06T01:40:42.491736+0000","last_fullsized":"2023-10-06T01:40:42.491736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486410+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:42.486410+0000","last_peered":"2023-10-06T01:40:42.486410+0000","last_clean":"2023-10-06T01:40:42.486410+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:42.486410+0000","last_undegraded":"2023-10-06T01:40:42.486410+0000","last_fullsized":"2023-10-06T01:40:42.486410+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.493970+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:40:42.493970+0000","last_peered":"2023-10-06T01:40:42.493970+0000","last_clean":"2023-10-06T01:40:42.493970+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:40:42.493970+0000","last_undegraded":"2023-10-06T01:40:42.493970+0000","last_fullsized":"2023-10-06T01:40:42.493970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491658+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:42.491658+0000","last_peered":"2023-10-06T01:40:42.491658+0000","last_clean":"2023-10-06T01:40:42.491658+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:42.491658+0000","last_undegraded":"2023-10-06T01:40:42.491658+0000","last_fullsized":"2023-10-06T01:40:42.491658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488369+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:40:42.488369+0000","last_peered":"2023-10-06T01:40:42.488369+0000","last_clean":"2023-10-06T01:40:42.488369+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:40:42.488369+0000","last_undegraded":"2023-10-06T01:40:42.488369+0000","last_fullsized":"2023-10-06T01:40:42.488369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488176+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:42.488176+0000","last_peered":"2023-10-06T01:40:42.488176+0000","last_clean":"2023-10-06T01:40:42.488176+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:42.488176+0000","last_undegraded":"2023-10-06T01:40:42.488176+0000","last_fullsized":"2023-10-06T01:40:42.488176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466278+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:42.466278+0000","last_peered":"2023-10-06T01:40:42.466278+0000","last_clean":"2023-10-06T01:40:42.466278+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:42.466278+0000","last_undegraded":"2023-10-06T01:40:42.466278+0000","last_fullsized":"2023-10-06T01:40:42.466278+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484499+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:40:42.484499+0000","last_peered":"2023-10-06T01:40:42.484499+0000","last_clean":"2023-10-06T01:40:42.484499+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:40:42.484499+0000","last_undegraded":"2023-10-06T01:40:42.484499+0000","last_fullsized":"2023-10-06T01:40:42.484499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466077+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:40:42.466077+0000","last_peered":"2023-10-06T01:40:42.466077+0000","last_clean":"2023-10-06T01:40:42.466077+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:40:42.466077+0000","last_undegraded":"2023-10-06T01:40:42.466077+0000","last_fullsized":"2023-10-06T01:40:42.466077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486405+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:42.486405+0000","last_peered":"2023-10-06T01:40:42.486405+0000","last_clean":"2023-10-06T01:40:42.486405+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:42.486405+0000","last_undegraded":"2023-10-06T01:40:42.486405+0000","last_fullsized":"2023-10-06T01:40:42.486405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491869+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:40:42.491869+0000","last_peered":"2023-10-06T01:40:42.491869+0000","last_clean":"2023-10-06T01:40:42.491869+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:40:42.491869+0000","last_undegraded":"2023-10-06T01:40:42.491869+0000","last_fullsized":"2023-10-06T01:40:42.491869+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488194+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:40:42.488194+0000","last_peered":"2023-10-06T01:40:42.488194+0000","last_clean":"2023-10-06T01:40:42.488194+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:40:42.488194+0000","last_undegraded":"2023-10-06T01:40:42.488194+0000","last_fullsized":"2023-10-06T01:40:42.488194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487367+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:40:42.487367+0000","last_peered":"2023-10-06T01:40:42.487367+0000","last_clean":"2023-10-06T01:40:42.487367+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:40:42.487367+0000","last_undegraded":"2023-10-06T01:40:42.487367+0000","last_fullsized":"2023-10-06T01:40:42.487367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.490387+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:42.490387+0000","last_peered":"2023-10-06T01:40:42.490387+0000","last_clean":"2023-10-06T01:40:42.490387+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:42.490387+0000","last_undegraded":"2023-10-06T01:40:42.490387+0000","last_fullsized":"2023-10-06T01:40:42.490387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.483166+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:40:42.483166+0000","last_peered":"2023-10-06T01:40:42.483166+0000","last_clean":"2023-10-06T01:40:42.483166+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:40:42.483166+0000","last_undegraded":"2023-10-06T01:40:42.483166+0000","last_fullsized":"2023-10-06T01:40:42.483166+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482813+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:42.482813+0000","last_peered":"2023-10-06T01:40:42.482813+0000","last_clean":"2023-10-06T01:40:42.482813+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:42.482813+0000","last_undegraded":"2023-10-06T01:40:42.482813+0000","last_fullsized":"2023-10-06T01:40:42.482813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.486018+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:40:42.486018+0000","last_peered":"2023-10-06T01:40:42.486018+0000","last_clean":"2023-10-06T01:40:42.486018+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:40:42.486018+0000","last_undegraded":"2023-10-06T01:40:42.486018+0000","last_fullsized":"2023-10-06T01:40:42.486018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.482560+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:42.482560+0000","last_peered":"2023-10-06T01:40:42.482560+0000","last_clean":"2023-10-06T01:40:42.482560+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:42.482560+0000","last_undegraded":"2023-10-06T01:40:42.482560+0000","last_fullsized":"2023-10-06T01:40:42.482560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466570+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:40:42.466570+0000","last_peered":"2023-10-06T01:40:42.466570+0000","last_clean":"2023-10-06T01:40:42.466570+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:40:42.466570+0000","last_undegraded":"2023-10-06T01:40:42.466570+0000","last_fullsized":"2023-10-06T01:40:42.466570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484527+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:40:42.484527+0000","last_peered":"2023-10-06T01:40:42.484527+0000","last_clean":"2023-10-06T01:40:42.484527+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:40:42.484527+0000","last_undegraded":"2023-10-06T01:40:42.484527+0000","last_fullsized":"2023-10-06T01:40:42.484527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488076+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:40:42.488076+0000","last_peered":"2023-10-06T01:40:42.488076+0000","last_clean":"2023-10-06T01:40:42.488076+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:40:42.488076+0000","last_undegraded":"2023-10-06T01:40:42.488076+0000","last_fullsized":"2023-10-06T01:40:42.488076+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.481475+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:40:42.481475+0000","last_peered":"2023-10-06T01:40:42.481475+0000","last_clean":"2023-10-06T01:40:42.481475+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:40:42.481475+0000","last_undegraded":"2023-10-06T01:40:42.481475+0000","last_fullsized":"2023-10-06T01:40:42.481475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.475995+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:40:42.475995+0000","last_peered":"2023-10-06T01:40:42.475995+0000","last_clean":"2023-10-06T01:40:42.475995+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:40:42.475995+0000","last_undegraded":"2023-10-06T01:40:42.475995+0000","last_fullsized":"2023-10-06T01:40:42.475995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.477776+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:40:42.477776+0000","last_peered":"2023-10-06T01:40:42.477776+0000","last_clean":"2023-10-06T01:40:42.477776+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:40:42.477776+0000","last_undegraded":"2023-10-06T01:40:42.477776+0000","last_fullsized":"2023-10-06T01:40:42.477776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.463503+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:42.463503+0000","last_peered":"2023-10-06T01:40:42.463503+0000","last_clean":"2023-10-06T01:40:42.463503+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:42.463503+0000","last_undegraded":"2023-10-06T01:40:42.463503+0000","last_fullsized":"2023-10-06T01:40:42.463503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487197+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:42.487197+0000","last_peered":"2023-10-06T01:40:42.487197+0000","last_clean":"2023-10-06T01:40:42.487197+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:42.487197+0000","last_undegraded":"2023-10-06T01:40:42.487197+0000","last_fullsized":"2023-10-06T01:40:42.487197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466131+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:42.466131+0000","last_peered":"2023-10-06T01:40:42.466131+0000","last_clean":"2023-10-06T01:40:42.466131+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:42.466131+0000","last_undegraded":"2023-10-06T01:40:42.466131+0000","last_fullsized":"2023-10-06T01:40:42.466131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484765+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:40:42.484765+0000","last_peered":"2023-10-06T01:40:42.484765+0000","last_clean":"2023-10-06T01:40:42.484765+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:40:42.484765+0000","last_undegraded":"2023-10-06T01:40:42.484765+0000","last_fullsized":"2023-10-06T01:40:42.484765+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.462312+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:40:42.462312+0000","last_peered":"2023-10-06T01:40:42.462312+0000","last_clean":"2023-10-06T01:40:42.462312+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:40:42.462312+0000","last_undegraded":"2023-10-06T01:40:42.462312+0000","last_fullsized":"2023-10-06T01:40:42.462312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.463248+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:40:42.463248+0000","last_peered":"2023-10-06T01:40:42.463248+0000","last_clean":"2023-10-06T01:40:42.463248+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:40:42.463248+0000","last_undegraded":"2023-10-06T01:40:42.463248+0000","last_fullsized":"2023-10-06T01:40:42.463248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484843+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:40:42.484843+0000","last_peered":"2023-10-06T01:40:42.484843+0000","last_clean":"2023-10-06T01:40:42.484843+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:40:42.484843+0000","last_undegraded":"2023-10-06T01:40:42.484843+0000","last_fullsized":"2023-10-06T01:40:42.484843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466387+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:42.466387+0000","last_peered":"2023-10-06T01:40:42.466387+0000","last_clean":"2023-10-06T01:40:42.466387+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:42.466387+0000","last_undegraded":"2023-10-06T01:40:42.466387+0000","last_fullsized":"2023-10-06T01:40:42.466387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466644+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:40:42.466644+0000","last_peered":"2023-10-06T01:40:42.466644+0000","last_clean":"2023-10-06T01:40:42.466644+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:40:42.466644+0000","last_undegraded":"2023-10-06T01:40:42.466644+0000","last_fullsized":"2023-10-06T01:40:42.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485372+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:42.485372+0000","last_peered":"2023-10-06T01:40:42.485372+0000","last_clean":"2023-10-06T01:40:42.485372+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:42.485372+0000","last_undegraded":"2023-10-06T01:40:42.485372+0000","last_fullsized":"2023-10-06T01:40:42.485372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460317+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:40:42.460317+0000","last_peered":"2023-10-06T01:40:42.460317+0000","last_clean":"2023-10-06T01:40:42.460317+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:40:42.460317+0000","last_undegraded":"2023-10-06T01:40:42.460317+0000","last_fullsized":"2023-10-06T01:40:42.460317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460448+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:40:42.460448+0000","last_peered":"2023-10-06T01:40:42.460448+0000","last_clean":"2023-10-06T01:40:42.460448+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:40:42.460448+0000","last_undegraded":"2023-10-06T01:40:42.460448+0000","last_fullsized":"2023-10-06T01:40:42.460448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466226+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:42.466226+0000","last_peered":"2023-10-06T01:40:42.466226+0000","last_clean":"2023-10-06T01:40:42.466226+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:42.466226+0000","last_undegraded":"2023-10-06T01:40:42.466226+0000","last_fullsized":"2023-10-06T01:40:42.466226+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.488075+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:42.488075+0000","last_peered":"2023-10-06T01:40:42.488075+0000","last_clean":"2023-10-06T01:40:42.488075+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:42.488075+0000","last_undegraded":"2023-10-06T01:40:42.488075+0000","last_fullsized":"2023-10-06T01:40:42.488075+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466902+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:42.466902+0000","last_peered":"2023-10-06T01:40:42.466902+0000","last_clean":"2023-10-06T01:40:42.466902+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:42.466902+0000","last_undegraded":"2023-10-06T01:40:42.466902+0000","last_fullsized":"2023-10-06T01:40:42.466902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466258+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:42.466258+0000","last_peered":"2023-10-06T01:40:42.466258+0000","last_clean":"2023-10-06T01:40:42.466258+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:42.466258+0000","last_undegraded":"2023-10-06T01:40:42.466258+0000","last_fullsized":"2023-10-06T01:40:42.466258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.487955+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:42.487955+0000","last_peered":"2023-10-06T01:40:42.487955+0000","last_clean":"2023-10-06T01:40:42.487955+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:42.487955+0000","last_undegraded":"2023-10-06T01:40:42.487955+0000","last_fullsized":"2023-10-06T01:40:42.487955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.485178+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:40:42.485178+0000","last_peered":"2023-10-06T01:40:42.485178+0000","last_clean":"2023-10-06T01:40:42.485178+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:40:42.485178+0000","last_undegraded":"2023-10-06T01:40:42.485178+0000","last_fullsized":"2023-10-06T01:40:42.485178+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.466120+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:42.466120+0000","last_peered":"2023-10-06T01:40:42.466120+0000","last_clean":"2023-10-06T01:40:42.466120+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:42.466120+0000","last_undegraded":"2023-10-06T01:40:42.466120+0000","last_fullsized":"2023-10-06T01:40:42.466120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.491814+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:42.491814+0000","last_peered":"2023-10-06T01:40:42.491814+0000","last_clean":"2023-10-06T01:40:42.491814+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:42.491814+0000","last_undegraded":"2023-10-06T01:40:42.491814+0000","last_fullsized":"2023-10-06T01:40:42.491814+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-06T01:40:43.565503+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:40:43.565503+0000","last_peered":"2023-10-06T01:40:43.565503+0000","last_clean":"2023-10-06T01:40:43.565503+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:40:43.565503+0000","last_undegraded":"2023-10-06T01:40:43.565503+0000","last_fullsized":"2023-10-06T01:40:43.565503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.476205+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:40:42.476205+0000","last_peered":"2023-10-06T01:40:42.476205+0000","last_clean":"2023-10-06T01:40:42.476205+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:40:42.476205+0000","last_undegraded":"2023-10-06T01:40:42.476205+0000","last_fullsized":"2023-10-06T01:40:42.476205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.484862+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:40:42.484862+0000","last_peered":"2023-10-06T01:40:42.484862+0000","last_clean":"2023-10-06T01:40:42.484862+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:40:42.484862+0000","last_undegraded":"2023-10-06T01:40:42.484862+0000","last_fullsized":"2023-10-06T01:40:42.484862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.460308+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:40:42.460308+0000","last_peered":"2023-10-06T01:40:42.460308+0000","last_clean":"2023-10-06T01:40:42.460308+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:40:42.460308+0000","last_undegraded":"2023-10-06T01:40:42.460308+0000","last_fullsized":"2023-10-06T01:40:42.460308+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":29,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.463452+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:42.463452+0000","last_peered":"2023-10-06T01:40:42.463452+0000","last_clean":"2023-10-06T01:40:42.463452+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:42.463452+0000","last_undegraded":"2023-10-06T01:40:42.463452+0000","last_fullsized":"2023-10-06T01:40:42.463452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.478667+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:40:42.478667+0000","last_peered":"2023-10-06T01:40:42.478667+0000","last_clean":"2023-10-06T01:40:42.478667+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:40:42.478667+0000","last_undegraded":"2023-10-06T01:40:42.478667+0000","last_fullsized":"2023-10-06T01:40:42.478667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.465585+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:42.465585+0000","last_peered":"2023-10-06T01:40:42.465585+0000","last_clean":"2023-10-06T01:40:42.465585+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:42.465585+0000","last_undegraded":"2023-10-06T01:40:42.465585+0000","last_fullsized":"2023-10-06T01:40:42.465585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T01:40:42.463641+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:42.463641+0000","last_peered":"2023-10-06T01:40:42.463641+0000","last_clean":"2023-10-06T01:40:42.463641+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:42.463641+0000","last_undegraded":"2023-10-06T01:40:42.463641+0000","last_fullsized":"2023-10-06T01:40:42.463641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":1011712,"data_stored":773265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.626}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":421888,"data_stored":183451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.433}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45100000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":1011712,"data_stored":773245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25700000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.23300000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.5},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":417792,"data_stored":182841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.29399999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":183012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.376},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.438}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.504}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":183417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.307}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.503}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.437},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":184343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":417792,"data_stored":182877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.372},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.5}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41499999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:40:51.376 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T01:40:51.376 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T01:40:51.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.528+0000 7f57ba89b700 1 Processor -- start 2023-10-06T01:40:51.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.528+0000 7f57ba89b700 1 -- start start 2023-10-06T01:40:51.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba89b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57b40cd400 0x7f57b40ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba89b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f57b40b2a30 0x7f57b40cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba89b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b40cc9b0 con 0x7f57b40b2660 2023-10-06T01:40:51.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba89b700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b40cf560 con 0x7f57b40b2a30 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba89b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b40cbda0 con 0x7f57b40cd400 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.529+0000 7f57ba09a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f57b40b2a30 0x7f57b40cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f57b40b2a30 0x7f57b40cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47046/0 (socket says 172.21.15.98:47046) 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 -- 172.21.15.98:0/3198538860 learned_addr learned my addr 172.21.15.98:0/3198538860 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57b9098700 1 --2- 172.21.15.98:0/3198538860 >> v2:172.21.15.184:3300/0 conn(0x7f57b40cd400 0x7f57b40ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 -- 172.21.15.98:0/3198538860 >> v2:172.21.15.184:3300/0 conn(0x7f57b40cd400 msgr2=0x7f57b40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 --2- 172.21.15.98:0/3198538860 >> v2:172.21.15.184:3300/0 conn(0x7f57b40cd400 0x7f57b40ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 -- 172.21.15.98:0/3198538860 >> v1:172.21.15.184:6789/0 conn(0x7f57b40b2660 legacy=0x7f57b40cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 -- 172.21.15.98:0/3198538860 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40cc080 con 0x7f57b40b2a30 2023-10-06T01:40:51.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57ba09a700 1 --2- 172.21.15.98:0/3198538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40b2a30 0x7f57b40cf020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f57b40b6310 tx=0x7f57ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35d23c8623f6b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57b8897700 1 -- 172.21.15.98:0/3198538860 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00e070 con 0x7f57b40b2a30 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57b8897700 1 -- 172.21.15.98:0/3198538860 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac00ba50 con 0x7f57b40b2a30 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.530+0000 7f57b8897700 1 -- 172.21.15.98:0/3198538860 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0092e0 con 0x7f57b40b2a30 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 -- 172.21.15.98:0/3198538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40b2a30 msgr2=0x7f57b40cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3198538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40b2a30 0x7f57b40cf020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f57b40b6310 tx=0x7f57ac009840 comp rx=0 tx=0).stop 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 -- 172.21.15.98:0/3198538860 shutdown_connections 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3198538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40b2a30 0x7f57b40cf020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3198538860 >> v2:172.21.15.184:3300/0 conn(0x7f57b40cd400 0x7f57b40ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 -- 172.21.15.98:0/3198538860 >> 172.21.15.98:0/3198538860 conn(0x7f57b4099560 msgr2=0x7f57b406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 -- 172.21.15.98:0/3198538860 shutdown_connections 2023-10-06T01:40:51.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.531+0000 7f57ba89b700 1 -- 172.21.15.98:0/3198538860 wait complete. 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.532+0000 7f57ba89b700 1 Processor -- start 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.532+0000 7f57ba89b700 1 -- start start 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.532+0000 7f57ba89b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57b40b2660 0x7f57b415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57b40b2a30 0x7f57b41544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b415bb50 con 0x7f57b40cd400 2023-10-06T01:40:51.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b415a570 con 0x7f57b40b2a30 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b415a850 con 0x7f57b40b2660 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47062/0 (socket says 172.21.15.98:47062) 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 -- 172.21.15.98:0/3380991863 learned_addr learned my addr 172.21.15.98:0/3380991863 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57b9098700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57b40b2a30 0x7f57b41544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57b9899700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57b40b2660 0x7f57b415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 -- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57b40b2660 msgr2=0x7f57b415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57b40b2660 0x7f57b415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 -- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57b40b2a30 msgr2=0x7f57b41544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57b40b2a30 0x7f57b41544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 -- 172.21.15.98:0/3380991863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009ae0 con 0x7f57b40cd400 2023-10-06T01:40:51.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba09a700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f57ac012000 tx=0x7f57ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac008e40 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac00be60 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b415ab30 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0266d0 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.533+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b415b0f0 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.534+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b415b2e0 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.535+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57ac003040 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.535+0000 7f57a9ffb700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 0x7f57980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.535+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f57ac0868a0 con 0x7f57b40cd400 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.535+0000 7f57b9899700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 0x7f57980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.535+0000 7f57b9899700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 0x7f57980693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.539+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57ac053870 con 0x7f57b40cd400 2023-10-06T01:40:51.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.798+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f57b405f270 con 0x7f57b40cd400 2023-10-06T01:40:51.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.801+0000 7f57a9ffb700 1 -- 172.21.15.98:0/3380991863 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f57ac056e90 con 0x7f57b40cd400 2023-10-06T01:40:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 msgr2=0x7f57980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 0x7f57980693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0009040 comp rx=0 tx=0).stop 2023-10-06T01:40:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 msgr2=0x7f57b4159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f57ac012000 tx=0x7f57ac018000 comp rx=0 tx=0).stop 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 shutdown_connections 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5798066f30 0x7f57980693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57b40cd400 0x7f57b4159c00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57b40b2660 0x7f57b415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 --2- 172.21.15.98:0/3380991863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57b40b2a30 0x7f57b41544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 >> 172.21.15.98:0/3380991863 conn(0x7f57b4099560 msgr2=0x7f57b40c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 shutdown_connections 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.803+0000 7f57ba89b700 1 -- 172.21.15.98:0/3380991863 wait complete. 2023-10-06T01:40:51.805 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2023-10-06T01:40:51.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:51.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.929+0000 7f09cad12700 1 Processor -- start 2023-10-06T01:40:51.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.929+0000 7f09cad12700 1 -- start start 2023-10-06T01:40:51.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09cad12700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09cad12700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09c40cb400 0x7f09c40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09cad12700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40cdb30 con 0x7f09c4159e30 2023-10-06T01:40:51.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09cad12700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c415e060 con 0x7f09c40cb400 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09cad12700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c415d4e0 con 0x7f09c40cd180 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09c92af700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f09c40cb400 0x7f09c40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:57588/0 (socket says 172.21.15.98:57588) 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09c3fff700 1 -- 172.21.15.98:0/2774619935 learned_addr learned my addr 172.21.15.98:0/2774619935 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.930+0000 7f09c37fe700 1 -- 172.21.15.98:0/2774619935 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752859935 0 0) 0x7f09c40cdb30 con 0x7f09c4159e30 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c37fe700 1 -- 172.21.15.98:0/2774619935 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b4003570 con 0x7f09c4159e30 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c92af700 1 -- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 msgr2=0x7f09c40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c92af700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c92af700 1 -- 172.21.15.98:0/2774619935 >> v1:172.21.15.184:6789/0 conn(0x7f09c4159e30 legacy=0x7f09c415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c92af700 1 -- 172.21.15.98:0/2774619935 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c415d7c0 con 0x7f09c40cb400 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c3fff700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c92af700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c40cb840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009e60 tx=0x7f09bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891bac25cb194b85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c37fe700 1 -- 172.21.15.98:0/2774619935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00e070 con 0x7f09c40cb400 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c37fe700 1 -- 172.21.15.98:0/2774619935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc00bab0 con 0x7f09c40cb400 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09c37fe700 1 -- 172.21.15.98:0/2774619935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00be50 con 0x7f09c40cb400 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09cad12700 1 -- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 msgr2=0x7f09c40cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09cad12700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c40cb840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009e60 tx=0x7f09bc009780 comp rx=0 tx=0).stop 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09cad12700 1 -- 172.21.15.98:0/2774619935 shutdown_connections 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09cad12700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c40cb840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.931+0000 7f09cad12700 1 --2- 172.21.15.98:0/2774619935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.935 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.932+0000 7f09cad12700 1 -- 172.21.15.98:0/2774619935 >> 172.21.15.98:0/2774619935 conn(0x7f09c4070810 msgr2=0x7f09c4070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:51.935 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.932+0000 7f09cad12700 1 -- 172.21.15.98:0/2774619935 shutdown_connections 2023-10-06T01:40:51.935 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.932+0000 7f09cad12700 1 -- 172.21.15.98:0/2774619935 wait complete. 2023-10-06T01:40:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 Processor -- start 2023-10-06T01:40:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 -- start start 2023-10-06T01:40:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c4156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.938 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40c4a10 con 0x7f09c4159e30 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40c4cf0 con 0x7f09c40cb400 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.933+0000 7f09cad12700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40c4fd0 con 0x7f09c40cd180 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47064/0 (socket says 172.21.15.98:47064) 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 -- 172.21.15.98:0/3527840207 learned_addr learned my addr 172.21.15.98:0/3527840207 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c3fff700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c8aae700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c4156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 -- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 msgr2=0x7f09c415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 -- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 msgr2=0x7f09c4156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c4156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 -- 172.21.15.98:0/3527840207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00094f0 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c92af700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b610 tx=0x7f09bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc012db0 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc025440 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc009a20 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc025800 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.934+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c40c5590 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.935+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c4064ad0 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.936+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09bc003050 con 0x7f09c4159e30 2023-10-06T01:40:51.940 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.936+0000 7f09c0ff9700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 0x7f09a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.936+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f09bc086240 con 0x7f09c4159e30 2023-10-06T01:40:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.936+0000 7f09c8aae700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 0x7f09a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.936+0000 7f09c8aae700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 0x7f09a80693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09b8007fd0 tx=0x7f09b8007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:51.941 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:51.940+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09bc053210 con 0x7f09c4159e30 2023-10-06T01:40:52.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.166+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c415daa0 con 0x7f09c4159e30 2023-10-06T01:40:52.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.167+0000 7f09c0ff9700 1 -- 172.21.15.98:0/3527840207 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f09bc05ae60 con 0x7f09c4159e30 2023-10-06T01:40:52.168 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:52.168 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/388888775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388888775},{"type":"v1","addr":"172.21.15.98:6836","nonce":388888775}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/582727435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":582727435},{"type":"v1","addr":"172.21.15.98:6837","nonce":582727435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2425522185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2425522185},{"type":"v1","addr":"172.21.15.184:6834","nonce":2425522185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:45.479675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1929325397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1929325397},{"type":"v1","addr":"172.21.15.184:6835","nonce":1929325397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T01:40:52.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 msgr2=0x7f09a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 0x7f09a80693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09b8007fd0 tx=0x7f09b8007d50 comp rx=0 tx=0).stop 2023-10-06T01:40:52.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 msgr2=0x7f09c415b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b610 tx=0x7f09bc019000 comp rx=0 tx=0).stop 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 shutdown_connections 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f09a8066f30 0x7f09a80693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c4159e30 0x7f09c415b650 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09c40cb400 0x7f09c4156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 --2- 172.21.15.98:0/3527840207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09c40cd180 0x7f09c415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 >> 172.21.15.98:0/3527840207 conn(0x7f09c4070810 msgr2=0x7f09c40cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 shutdown_connections 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.169+0000 7f09cad12700 1 -- 172.21.15.98:0/3527840207 wait complete. 2023-10-06T01:40:52.171 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2023-10-06T01:40:52.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:40:41.845111+0000', 'modified': '2023-10-06T01:40:45.479675+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5133': {'gid': 5133, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/1929325397', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6833', 'nonce': 1929325397}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1929325397}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T01:40:52.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T01:40:52.585 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T01:40:52.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 Processor -- start 2023-10-06T01:40:52.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 -- start start 2023-10-06T01:40:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f311c1595a0 0x7f311c0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 0x7f311c0aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b07a0 0x7f311c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0b0ca0 con 0x7f311c1595a0 2023-10-06T01:40:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0b0f80 con 0x7f311c0b49e0 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.700+0000 7f3122b17700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0b1260 con 0x7f311c0b07a0 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f311c1595a0 0x7f311c0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f311c1595a0 0x7f311c0aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55202/0 (socket says 172.21.15.98:55202) 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 -- 172.21.15.98:0/3921708006 learned_addr learned my addr 172.21.15.98:0/3921708006 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31210b4700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b07a0 0x7f311c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f311bfff700 1 --2- 172.21.15.98:0/3921708006 >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 0x7f311c0aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 -- 172.21.15.98:0/3921708006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b07a0 msgr2=0x7f311c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b07a0 0x7f311c0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 -- 172.21.15.98:0/3921708006 >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 msgr2=0x7f311c0aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 --2- 172.21.15.98:0/3921708006 >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 0x7f311c0aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 -- 172.21.15.98:0/3921708006 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0b1540 con 0x7f311c1595a0 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f311bfff700 1 --2- 172.21.15.98:0/3921708006 >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 0x7f311c0aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f31208b3700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0aa570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f310c009f70 tx=0x7f310c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74e485d6bac82ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f311b7fe700 1 -- 172.21.15.98:0/3921708006 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00e070 con 0x7f311c1595a0 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f311b7fe700 1 -- 172.21.15.98:0/3921708006 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c00ba90 con 0x7f311c1595a0 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.701+0000 7f311b7fe700 1 -- 172.21.15.98:0/3921708006 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c0092e0 con 0x7f311c1595a0 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 -- 172.21.15.98:0/3921708006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 msgr2=0x7f311c0aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0aa570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f310c009f70 tx=0x7f310c009840 comp rx=0 tx=0).stop 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 -- 172.21.15.98:0/3921708006 shutdown_connections 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b07a0 0x7f311c0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 --2- 172.21.15.98:0/3921708006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0aa570 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 --2- 172.21.15.98:0/3921708006 >> v2:172.21.15.184:3300/0 conn(0x7f311c0b49e0 0x7f311c0aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 -- 172.21.15.98:0/3921708006 >> 172.21.15.98:0/3921708006 conn(0x7f311c099830 msgr2=0x7f311c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 -- 172.21.15.98:0/3921708006 shutdown_connections 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.702+0000 7f3122b17700 1 -- 172.21.15.98:0/3921708006 wait complete. 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.703+0000 7f3122b17700 1 Processor -- start 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.703+0000 7f3122b17700 1 -- start start 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f311c0b07a0 0x7f311c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 0x7f311c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0afa20 con 0x7f311c1595a0 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c09d3c0 con 0x7f311c0b07a0 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c09d6a0 con 0x7f311c0b49e0 2023-10-06T01:40:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55206/0 (socket says 172.21.15.98:55206) 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 -- 172.21.15.98:0/3019469415 learned_addr learned my addr 172.21.15.98:0/3019469415 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f311bfff700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 0x7f311c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31208b3700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f311c0b07a0 0x7f311c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 -- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 msgr2=0x7f311c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 0x7f311c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 -- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f311c0b07a0 msgr2=0x7f311c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f311c0b07a0 0x7f311c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 -- 172.21.15.98:0/3019469415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c009ae0 con 0x7f311c1595a0 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f311bfff700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 0x7f311c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f31210b4700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f311400d890 tx=0x7f3114008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3118ff9700 1 -- 172.21.15.98:0/3019469415 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140125f0 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3118ff9700 1 -- 172.21.15.98:0/3019469415 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114012770 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c09d980 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3118ff9700 1 -- 172.21.15.98:0/3019469415 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114012b30 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.704+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c09c630 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.706+0000 7f3118ff9700 1 -- 172.21.15.98:0/3019469415 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f311401e460 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.706+0000 7f3118ff9700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 0x7f30fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.706+0000 7f3118ff9700 1 -- 172.21.15.98:0/3019469415 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3114084e10 con 0x7f311c1595a0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.706+0000 7f31208b3700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 0x7f30fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.707+0000 7f31208b3700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 0x7f30fc0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f310c009880 tx=0x7f310c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.708+0000 7f312be0fb80 1 Processor -- start 2023-10-06T01:40:52.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.708+0000 7f312be0fb80 1 -- start start 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf151c00 0x556edf152050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf058a70 con 0x556edf101840 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf056270 con 0x556edf151c00 2023-10-06T01:40:52.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edf052060 con 0x556edf102cc0 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55208/0 (socket says 172.21.15.98:55208) 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 -- 172.21.15.98:0/2211119217 learned_addr learned my addr 172.21.15.98:0/2211119217 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 -- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 msgr2=0x556edf1516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 -- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf151c00 msgr2=0x556edf152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf151c00 0x556edf152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 -- 172.21.15.98:0/2211119217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556edeeec270 con 0x556edf101840 2023-10-06T01:40:52.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f31208b3700 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f310c00bc80 tx=0x7f310c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c45e315af7ef6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f3111ffb700 1 -- 172.21.15.98:0/2211119217 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c0076e0 con 0x556edf101840 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f3111ffb700 1 -- 172.21.15.98:0/2211119217 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c021070 con 0x556edf101840 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.709+0000 7f3111ffb700 1 -- 172.21.15.98:0/2211119217 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c007a50 con 0x556edf101840 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 -- 172.21.15.98:0/2211119217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 msgr2=0x556edf101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f310c00bc80 tx=0x7f310c01f070 comp rx=0 tx=0).stop 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 -- 172.21.15.98:0/2211119217 shutdown_connections 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf151c00 0x556edf152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf101c10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 --2- 172.21.15.98:0/2211119217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 -- 172.21.15.98:0/2211119217 >> 172.21.15.98:0/2211119217 conn(0x556edf103160 msgr2=0x556edf103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 -- 172.21.15.98:0/2211119217 shutdown_connections 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.710+0000 7f312be0fb80 1 -- 172.21.15.98:0/2211119217 wait complete. 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.711+0000 7f312be0fb80 1 -- start start 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.711+0000 7f312be0fb80 1 Processor -- start 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf158990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edeffd890 con 0x556edf102cc0 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf018ea0 con 0x556edf101840 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edeed4e30 con 0x556edf151c00 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 10 client.0 _ll_register_callbacks cb 0x556edf0c4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55222/0 (socket says 172.21.15.98:55222) 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 -- 172.21.15.98:0/3226328622 learned_addr learned my addr 172.21.15.98:0/3226328622 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f31210b4700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f31208b3700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 -- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 msgr2=0x556edf142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 -- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 msgr2=0x556edf158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 -- 172.21.15.98:0/3226328622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c0b1540 con 0x556edf102cc0 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f311bfff700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f311c088e20 tx=0x7f3104008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fbbf3df7a56eee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f30faffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104013450 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f31040135b0 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556edf0f6940 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3104013980 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 10 client.5346 fetch_fsmap learned FSMap version 17 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x556edee8efe0 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.712+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556edf0f6940 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3104011070 con 0x556edf102cc0 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 10 client.5346 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 10 client.5346 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T01:40:52.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 20 client.5346 populate_metadata read hostname 'smithi098' 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf12c130 0x556edf12c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 4 client.5346 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 --> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556edf0d3870 con 0x556edf12c130 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f31208b3700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf12c130 0x556edf12c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f31208b3700 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf12c130 0x556edf12c500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740e1b0bc3ecb4c2 server_cookie=a97334116ef4bc90 in_seq=0 out_seq=0 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.713+0000 7f30faffd700 10 client.5346 ms_handle_connect on v2:172.21.15.184:6833/1929325397 2023-10-06T01:40:52.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f30faffd700 1 -- 172.21.15.98:0/3226328622 <== mds.0 v2:172.21.15.184:6833/1929325397 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556edf0d3870 con 0x556edf12c130 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f30faffd700 10 client.5346 handle_command_reply: tid=0 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 client.5346 shutdown 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 msgr2=0x556edf1423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f311c088e20 tx=0x7f3104008040 comp rx=0 tx=0).stop 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 shutdown_connections 2023-10-06T01:40:52.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf12c130 0x556edf12c500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 --2- 172.21.15.98:0/3226328622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf102cc0 0x556edf1423e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f30faffd700 0 client.5346 ms_handle_reset on v2:172.21.15.184:6833/1929325397 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 >> 172.21.15.98:0/3226328622 conn(0x556edf103160 msgr2=0x556edf12ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 shutdown_connections 2023-10-06T01:40:52.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 1 -- 172.21.15.98:0/3226328622 wait complete. 2023-10-06T01:40:52.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.714+0000 7f312be0fb80 20 client.5346 trim_cache size 0 max 16384 2023-10-06T01:40:52.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 Processor -- start 2023-10-06T01:40:52.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 -- start start 2023-10-06T01:40:52.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf102cc0 0x556edf1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.739 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf127940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf0fc6b0 con 0x556edf101840 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf12c5a0 con 0x556edf102cc0 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edf0e1380 con 0x556edf151c00 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f31208b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f31208b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55232/0 (socket says 172.21.15.98:55232) 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.731+0000 7f31208b3700 1 -- 172.21.15.98:0/906575090 learned_addr learned my addr 172.21.15.98:0/906575090 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 -- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 msgr2=0x556edf127940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf127940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f311bfff700 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf102cc0 0x556edf1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.740 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 -- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf102cc0 msgr2=0x556edf1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf102cc0 0x556edf1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 -- 172.21.15.98:0/906575090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556edf020780 con 0x556edf101840 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31208b3700 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f310c00d010 tx=0x7f310c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcfc944460f000e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31127fc700 1 -- 172.21.15.98:0/906575090 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c039040 con 0x556edf101840 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31127fc700 1 -- 172.21.15.98:0/906575090 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c0076e0 con 0x556edf101840 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f31127fc700 1 -- 172.21.15.98:0/906575090 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c007890 con 0x556edf101840 2023-10-06T01:40:52.741 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 -- 172.21.15.98:0/906575090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 msgr2=0x556edf158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f310c00d010 tx=0x7f310c035040 comp rx=0 tx=0).stop 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 -- 172.21.15.98:0/906575090 shutdown_connections 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf151c00 0x556edf127940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf101840 0x556edf158360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 --2- 172.21.15.98:0/906575090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf102cc0 0x556edf1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.732+0000 7f312be0fb80 1 -- 172.21.15.98:0/906575090 >> 172.21.15.98:0/906575090 conn(0x556edf103160 msgr2=0x556edf1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.733+0000 7f312be0fb80 1 -- 172.21.15.98:0/906575090 shutdown_connections 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.733+0000 7f312be0fb80 1 -- 172.21.15.98:0/906575090 wait complete. 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.733+0000 7f312be0fb80 1 -- start start 2023-10-06T01:40:52.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.733+0000 7f312be0fb80 1 Processor -- start 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf1aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf1a84d0 con 0x556edf151c00 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edf14cca0 con 0x556edf101840 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edf0fa410 con 0x556edf102cc0 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f312be0fb80 10 client.0 _ll_register_callbacks cb 0x556edf0c4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55236/0 (socket says 172.21.15.98:55236) 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 -- 172.21.15.98:0/992632663 learned_addr learned my addr 172.21.15.98:0/992632663 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.743 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31208b3700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf1aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f311bfff700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 -- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 msgr2=0x556edf1a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 -- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 msgr2=0x556edf1aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf1aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31210b4700 1 -- 172.21.15.98:0/992632663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556edeeec270 con 0x556edf151c00 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.734+0000 7f31208b3700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf1aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f31210b4700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3114051fa0 tx=0x7f3114020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bb597e9af5f0d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311405ca40 con 0x556edf151c00 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f310c021070 con 0x556edf151c00 2023-10-06T01:40:52.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3114055d00 con 0x556edf151c00 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3114054f90 con 0x556edf151c00 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 10 client.5352 fetch_fsmap learned FSMap version 17 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x556edf007470 con 0x556edf151c00 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114060030 con 0x556edf151c00 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f311405c5b0 con 0x556edf151c00 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 10 client.5352 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 10 client.5352 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 20 client.5352 populate_metadata read hostname 'smithi098' 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf103650 0x556edf103a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 4 client.5352 mds_command: new command op to 5133 tid=0[{"prefix": "status"}] 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 --> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556edeff4e00 con 0x556edf103650 2023-10-06T01:40:52.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.735+0000 7f31208b3700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf103650 0x556edf103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f30faffd700 10 client.5352 ms_handle_connect on v2:172.21.15.184:6833/1929325397 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f31208b3700 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf103650 0x556edf103a20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1342361a72d38891 server_cookie=550da6ce924f4108 in_seq=0 out_seq=0 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f30faffd700 1 -- 172.21.15.98:0/992632663 <== mds.0 v2:172.21.15.184:6833/1929325397 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556edeff4e00 con 0x556edf103650 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f30faffd700 10 client.5352 handle_command_reply: tid=0 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 client.5352 shutdown 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 msgr2=0x556edf14bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3114051fa0 tx=0x7f3114020040 comp rx=0 tx=0).stop 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 shutdown_connections 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:6833/1929325397,v1:172.21.15.184:6835/1929325397] conn(0x556edf103650 0x556edf103a20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556edf151c00 0x556edf14bf10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f30faffd700 0 client.5352 ms_handle_reset on v2:172.21.15.184:6833/1929325397 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556edf101840 0x556edf1aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 --2- 172.21.15.98:0/992632663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556edf102cc0 0x556edf1a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 >> 172.21.15.98:0/992632663 conn(0x556edf103160 msgr2=0x556edf141da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 shutdown_connections 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 1 -- 172.21.15.98:0/992632663 wait complete. 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.736+0000 7f312be0fb80 20 client.5352 trim_cache size 0 max 16384 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "c6dd341a-820f-4e37-8897-3250639f93a1", 2023-10-06T01:40:52.747 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5133, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 10.269043934000001, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 17, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 36, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 36, 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 18.168242393 2023-10-06T01:40:52.748 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 msgr2=0x7f30fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 0x7f30fc0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f310c009880 tx=0x7f310c018000 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 msgr2=0x7f311c0cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f311400d890 tx=0x7f3114008040 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 shutdown_connections 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f30fc066f30 0x7f30fc0693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f311c1595a0 0x7f311c0cee70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f311c0b07a0 0x7f311c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 --2- 172.21.15.98:0/3019469415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f311c0b49e0 0x7f311c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.750 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 >> 172.21.15.98:0/3019469415 conn(0x7f311c099830 msgr2=0x7f311c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 shutdown_connections 2023-10-06T01:40:52.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.737+0000 7f3122b17700 1 -- 172.21.15.98:0/3019469415 wait complete. 2023-10-06T01:40:52.751 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:40:52.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.864+0000 7fb49f8fb700 1 Processor -- start 2023-10-06T01:40:52.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.864+0000 7fb49f8fb700 1 -- start start 2023-10-06T01:40:52.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49f8fb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb498159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49f8fb700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb4980ca6b0 0x7fb4980caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49f8fb700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb498068d50 con 0x7fb4980cb630 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49f8fb700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb498154060 con 0x7fb4980ca6b0 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49f8fb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49815a4a0 con 0x7fb498159b10 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb4980ca6b0 0x7fb4980caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb4980ca6b0 0x7fb4980caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55248/0 (socket says 172.21.15.98:55248) 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 -- 172.21.15.98:0/271820787 learned_addr learned my addr 172.21.15.98:0/271820787 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 -- 172.21.15.98:0/271820787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 msgr2=0x7fb498159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49ce96700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb498159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb498159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 -- 172.21.15.98:0/271820787 >> v1:172.21.15.184:6789/0 conn(0x7fb4980cb630 legacy=0x7fb4980691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 -- 172.21.15.98:0/271820787 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49815a6b0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49ce96700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb498159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:40:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb49de98700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb4980caaf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb4900099b0 tx=0x7fb490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1480cfb2d784f96d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.865+0000 7fb48ffff700 1 -- 172.21.15.98:0/271820787 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49000b860 con 0x7fb4980ca6b0 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb48ffff700 1 -- 172.21.15.98:0/271820787 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49000b9c0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb48ffff700 1 -- 172.21.15.98:0/271820787 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49000bd60 con 0x7fb4980ca6b0 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 -- 172.21.15.98:0/271820787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 msgr2=0x7fb4980caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb4980caaf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb4900099b0 tx=0x7fb490009250 comp rx=0 tx=0).stop 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 -- 172.21.15.98:0/271820787 shutdown_connections 2023-10-06T01:40:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb4980caaf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/271820787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb498159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 -- 172.21.15.98:0/271820787 >> 172.21.15.98:0/271820787 conn(0x7fb498070810 msgr2=0x7fb498070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 -- 172.21.15.98:0/271820787 shutdown_connections 2023-10-06T01:40:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.866+0000 7fb49f8fb700 1 -- 172.21.15.98:0/271820787 wait complete. 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.867+0000 7fb49f8fb700 1 Processor -- start 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.867+0000 7fb49f8fb700 1 -- start start 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4980cb630 0x7fb498154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb4980690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb498155740 con 0x7fb4980ca6b0 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb498154ea0 con 0x7fb4980cb630 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49f8fb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb498155180 con 0x7fb498159b10 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55252/0 (socket says 172.21.15.98:55252) 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 -- 172.21.15.98:0/2305595433 learned_addr learned my addr 172.21.15.98:0/2305595433 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49de98700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb4980690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49ce96700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4980cb630 0x7fb498154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 -- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 msgr2=0x7fb4980690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb4980690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 -- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4980cb630 msgr2=0x7fb498154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4980cb630 0x7fb498154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.868+0000 7fb49d697700 1 -- 172.21.15.98:0/2305595433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900094f0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb49d697700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb49400cb10 tx=0x7fb4940017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400ce20 con 0x7fb4980ca6b0 2023-10-06T01:40:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49400a7e0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400aba0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb498155460 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.869+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb498151eb0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.870+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4980b2110 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.870+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb494003030 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.870+0000 7fb48d7fa700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 0x7fb47c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.871+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb4940847a0 con 0x7fb4980ca6b0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.871+0000 7fb49ce96700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 0x7fb47c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:52.878 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.871+0000 7fb49ce96700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 0x7fb47c0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:52.875+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4980b2110 con 0x7fb4980ca6b0 2023-10-06T01:40:53.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.100+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4980b2110 con 0x7fb4980ca6b0 2023-10-06T01:40:53.102 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.100+0000 7fb48d7fa700 1 -- 172.21.15.98:0/2305595433 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4980b2110 con 0x7fb4980ca6b0 2023-10-06T01:40:53.102 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 msgr2=0x7fb47c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 0x7fb47c0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488008040 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 msgr2=0x7fb498151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb49400cb10 tx=0x7fb4940017b0 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 shutdown_connections 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb47c066f30 0x7fb47c0693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb498159b10 0x7fb4980690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4980ca6b0 0x7fb498151970 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 --2- 172.21.15.98:0/2305595433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4980cb630 0x7fb498154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 >> 172.21.15.98:0/2305595433 conn(0x7fb498070810 msgr2=0x7fb4981537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:53.104 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 shutdown_connections 2023-10-06T01:40:53.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:53.102+0000 7fb49f8fb700 1 -- 172.21.15.98:0/2305595433 wait complete. 2023-10-06T01:40:53.115 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T01:40:53.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:40:53.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2023-10-06T01:40:53.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T01:40:53.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:53.116 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T01:40:53.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T01:40:53.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:40:53.116 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:40:53.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.116 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:53.132 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41885sec preferred_lft 41885sec 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2023-10-06T01:40:53.133 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c0ec:e9ff:fe71:9101/64 scope link 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::4442:48ff:fe3d:c359/64 scope link 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.134 INFO:teuthology.orchestra.run.smithi098.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:40:53.135 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether a6:76:1f:1a:8e:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T01:40:53.135 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::a476:1fff:fe1a:8e84/64 scope link 2023-10-06T01:40:53.135 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:40:53.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.135 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:40:53.195 INFO:teuthology.orchestra.run.smithi098.stdout:1 2023-10-06T01:40:53.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.196 DEBUG:teuthology.orchestra.run.smithi098:> route 2023-10-06T01:40:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2023-10-06T01:40:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:40:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:40:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:40:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:40:53.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:40:53.253 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:40:53.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.352 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:40:53.367 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T01:40:53.367 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T01:40:53.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:40:53.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2023-10-06T01:40:53.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:53.432 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:53.486 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:40:53.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:40:53.486 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:53.540 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:40:53.540 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:40:53.540 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:53.619 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:53.619 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:53.619 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:53.628 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:53.628 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:53.628 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:53.628 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:53.628 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:40:53.651 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:40:53.652 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:40:53.652 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:40:53.653 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:40:53.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:53.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:53.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:53.718 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-06T01:40:53.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:53.778 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-10-06T01:40:53.904 INFO:teuthology.orchestra.run.smithi098.stderr:6+0 records in 2023-10-06T01:40:53.905 INFO:teuthology.orchestra.run.smithi098.stderr:6+0 records out 2023-10-06T01:40:53.905 INFO:teuthology.orchestra.run.smithi098.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0704245 s, 89.3 MB/s 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> import os 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> import stat 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> import json 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> import sys 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:40:53.906 DEBUG:teuthology.orchestra.run.smithi098:> try: 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> except OSError as e: 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> sys.exit(e.errno) 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> print(json.dumps( 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> indent=2)) 2023-10-06T01:40:53.907 DEBUG:teuthology.orchestra.run.smithi098:> ' 2023-10-06T01:40:53.975 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:40:53.975 INFO:teuthology.orchestra.run.smithi098.stdout: "st_mode": 33204, 2023-10-06T01:40:53.975 INFO:teuthology.orchestra.run.smithi098.stdout: "st_ino": 1099511627777, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_dev": 46, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_nlink": 1, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_uid": 1000, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_gid": 1275, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_size": 6291456, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_atime": 1696556453.8329904, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_mtime": 1696556453.86299, 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout: "st_ctime": 1696556453.86299 2023-10-06T01:40:53.976 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:40:54.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T01:40:54.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:40:54.160 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T01:40:54.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:40:54.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:40:54.266 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T01:40:54.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:40:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.441+0000 7f99da3bd700 1 Processor -- start 2023-10-06T01:40:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.441+0000 7f99da3bd700 1 -- start start 2023-10-06T01:40:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99da3bd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f99d4159480 0x7f99d40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99da3bd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99d40b3610 0x7f99d40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99da3bd700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d40cb7e0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99da3bd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d40b08a0 con 0x7f99d40b3610 2023-10-06T01:40:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99da3bd700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d40cb190 con 0x7f99d4159480 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d8bba700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99d40b3610 0x7f99d40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d9bbc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f99d40cc1b0 0x7f99d40b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:34886/0 (socket says 172.21.15.98:34886) 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d9bbc700 1 -- 172.21.15.98:0/1244347711 learned_addr learned my addr 172.21.15.98:0/1244347711 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d93bb700 1 --2- 172.21.15.98:0/1244347711 >> v2:172.21.15.184:3300/0 conn(0x7f99d4159480 0x7f99d40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d8bba700 1 -- 172.21.15.98:0/1244347711 >> v2:172.21.15.184:3300/0 conn(0x7f99d4159480 msgr2=0x7f99d40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d8bba700 1 --2- 172.21.15.98:0/1244347711 >> v2:172.21.15.184:3300/0 conn(0x7f99d4159480 0x7f99d40b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d8bba700 1 -- 172.21.15.98:0/1244347711 >> v1:172.21.15.98:6789/0 conn(0x7f99d40cc1b0 legacy=0x7f99d40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99cbfff700 1 -- 172.21.15.98:0/1244347711 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039589755 0 0) 0x7f99d40cb7e0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.442+0000 7f99d8bba700 1 -- 172.21.15.98:0/1244347711 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d40cb470 con 0x7f99d40b3610 2023-10-06T01:40:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99d8bba700 1 --2- 172.21.15.98:0/1244347711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40b3610 0x7f99d40cbc70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f99d406fa30 tx=0x7f99c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9d777997abe01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99cbfff700 1 -- 172.21.15.98:0/1244347711 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011630 con 0x7f99d40b3610 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99cbfff700 1 -- 172.21.15.98:0/1244347711 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c4011790 con 0x7f99d40b3610 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99cbfff700 1 -- 172.21.15.98:0/1244347711 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011b30 con 0x7f99d40b3610 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 -- 172.21.15.98:0/1244347711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40b3610 msgr2=0x7f99d40cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1244347711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40b3610 0x7f99d40cbc70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f99d406fa30 tx=0x7f99c40091a0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 -- 172.21.15.98:0/1244347711 shutdown_connections 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1244347711 >> v2:172.21.15.184:3300/0 conn(0x7f99d4159480 0x7f99d40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1244347711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40b3610 0x7f99d40cbc70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 -- 172.21.15.98:0/1244347711 >> 172.21.15.98:0/1244347711 conn(0x7f99d406ed20 msgr2=0x7f99d406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 -- 172.21.15.98:0/1244347711 shutdown_connections 2023-10-06T01:40:54.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.443+0000 7f99da3bd700 1 -- 172.21.15.98:0/1244347711 wait complete. 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.444+0000 7f99da3bd700 1 Processor -- start 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.444+0000 7f99da3bd700 1 -- start start 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99d40b3610 0x7f99d4169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99d4159480 0x7f99d40cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d416a640 con 0x7f99d40cc1b0 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d416a940 con 0x7f99d40b3610 2023-10-06T01:40:54.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99da3bd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d415b750 con 0x7f99d4159480 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99d8bba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99d8bba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55260/0 (socket says 172.21.15.98:55260) 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.445+0000 7f99d8bba700 1 -- 172.21.15.98:0/1725653098 learned_addr learned my addr 172.21.15.98:0/1725653098 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d93bb700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99d40b3610 0x7f99d4169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d9bbc700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99d4159480 0x7f99d40cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 -- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99d4159480 msgr2=0x7f99d40cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99d4159480 0x7f99d40cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 -- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99d40b3610 msgr2=0x7f99d4169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99d40b3610 0x7f99d4169a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 -- 172.21.15.98:0/1725653098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009420 con 0x7f99d40cc1b0 2023-10-06T01:40:54.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99d8bba700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f99d4069190 tx=0x7f99c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011b10 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c40253f0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d415b160 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4025790 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.446+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d40cfb10 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.447+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d40b60c0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.447+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99c4025b80 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.448+0000 7f99c97fa700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 0x7f99b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.448+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f99c40860c0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.448+0000 7f99d93bb700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 0x7f99b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.448+0000 7f99d93bb700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 0x7f99b80693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009c10 tx=0x7f99d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.453 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.452+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99c4053090 con 0x7f99d40cc1b0 2023-10-06T01:40:54.682 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.680+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99d405f2d0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.681+0000 7f99c97fa700 1 -- 172.21.15.98:0/1725653098 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f99c40566b0 con 0x7f99d40cc1b0 2023-10-06T01:40:54.683 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:54.683 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/388888775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":388888775},{"type":"v1","addr":"172.21.15.98:6836","nonce":388888775}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/582727435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":582727435},{"type":"v1","addr":"172.21.15.98:6837","nonce":582727435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2425522185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2425522185},{"type":"v1","addr":"172.21.15.184:6834","nonce":2425522185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:45.479675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1929325397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1929325397},{"type":"v1","addr":"172.21.15.184:6835","nonce":1929325397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 msgr2=0x7f99b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 0x7f99b80693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009c10 tx=0x7f99d0008040 comp rx=0 tx=0).stop 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 msgr2=0x7f99d40cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f99d4069190 tx=0x7f99c4017000 comp rx=0 tx=0).stop 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 shutdown_connections 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f99b8066f30 0x7f99b80693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99d4159480 0x7f99d40cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99d40b3610 0x7f99d4169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 --2- 172.21.15.98:0/1725653098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d40cc1b0 0x7f99d40cec60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 >> 172.21.15.98:0/1725653098 conn(0x7f99d406ed20 msgr2=0x7f99d4156870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 shutdown_connections 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.683+0000 7f99da3bd700 1 -- 172.21.15.98:0/1725653098 wait complete. 2023-10-06T01:40:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2023-10-06T01:40:54.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-10-06T01:40:54.854 INFO:teuthology.orchestra.run.smithi184.stdout:{"message":"","return_code":0} 2023-10-06T01:40:54.854 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T01:40:54.854 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T01:40:54.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.968+0000 7f2619ae8700 1 Processor -- start 2023-10-06T01:40:54.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.968+0000 7f2619ae8700 1 -- start start 2023-10-06T01:40:54.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f2619ae8700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f26140b07a0 0x7f26140ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f2619ae8700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140b0ca0 con 0x7f26141595a0 2023-10-06T01:40:54.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f2619ae8700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140b0f80 con 0x7f26140b49e0 2023-10-06T01:40:54.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f2619ae8700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140ab390 con 0x7f26140b07a0 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f26137fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f26141595a0 0x7f26140aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:34892/0 (socket says 172.21.15.98:34892) 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f26137fe700 1 -- 172.21.15.98:0/2657237696 learned_addr learned my addr 172.21.15.98:0/2657237696 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f2613fff700 1 --2- 172.21.15.98:0/2657237696 >> v2:172.21.15.184:3300/0 conn(0x7f26140b07a0 0x7f26140ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741547071 0 0) 0x7f26140b0ca0 con 0x7f26141595a0 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.969+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25fc003570 con 0x7f26141595a0 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 628488181 0 0) 0x7f25fc003570 con 0x7f26141595a0 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 >> v2:172.21.15.184:3300/0 conn(0x7f26140b07a0 msgr2=0x7f26140ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 --2- 172.21.15.98:0/2657237696 >> v2:172.21.15.184:3300/0 conn(0x7f26140b07a0 0x7f26140ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 >> v1:172.21.15.184:6789/0 conn(0x7f26140b49e0 legacy=0x7f26140aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140b1260 con 0x7f26141595a0 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f26040015f0 con 0x7f26141595a0 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 >> v1:172.21.15.98:6789/0 conn(0x7f26141595a0 legacy=0x7f26140aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 0x7f25fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc004810 0x7f25fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25fc0051c0 0x7f25fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140ab390 con 0x7f25fc003a60 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc00c0d0 con 0x7f25fc004810 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc00c290 con 0x7f25fc0051c0 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 0x7f25fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f2612ffd700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc004810 0x7f25fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f2613fff700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25fc0051c0 0x7f25fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 -- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25fc0051c0 msgr2=0x7f25fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25fc0051c0 0x7f25fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 -- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc004810 msgr2=0x7f25fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc004810 0x7f25fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f2613fff700 1 -- 172.21.15.98:0/2657237696 reap_dead start 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 -- 172.21.15.98:0/2657237696 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc003570 con 0x7f25fc003a60 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f2612ffd700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc004810 0x7f25fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26137fe700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 0x7f25fc003e70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f260400c3c0 tx=0x7f2604007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110ca25b64bc1417 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.970+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400cc10 con 0x7f25fc003a60 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400cd90 con 0x7f25fc003a60 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f26127fc700 1 -- 172.21.15.98:0/2657237696 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604007320 con 0x7f25fc003a60 2023-10-06T01:40:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 -- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 msgr2=0x7f25fc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 0x7f25fc003e70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f260400c3c0 tx=0x7f2604007920 comp rx=0 tx=0).stop 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 -- 172.21.15.98:0/2657237696 shutdown_connections 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 --2- 172.21.15.98:0/2657237696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25fc003a60 0x7f25fc003e70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 -- 172.21.15.98:0/2657237696 >> 172.21.15.98:0/2657237696 conn(0x7f2614099830 msgr2=0x7f261406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 -- 172.21.15.98:0/2657237696 shutdown_connections 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.971+0000 7f2619ae8700 1 -- 172.21.15.98:0/2657237696 wait complete. 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.972+0000 7f2619ae8700 1 Processor -- start 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.972+0000 7f2619ae8700 1 -- start start 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc003a60 0x7f2614158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140a8c90 0x7f26140a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140a9650 con 0x7f26140a87c0 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261415af50 con 0x7f25fc003a60 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2619ae8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261415a380 con 0x7f26140a8c90 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55280/0 (socket says 172.21.15.98:55280) 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 -- 172.21.15.98:0/264589771 learned_addr learned my addr 172.21.15.98:0/264589771 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2613fff700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140a8c90 0x7f26140a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f26137fe700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc003a60 0x7f2614158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 -- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140a8c90 msgr2=0x7f26140a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140a8c90 0x7f26140a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 -- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc003a60 msgr2=0x7f2614158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc003a60 0x7f2614158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.973+0000 7f2612ffd700 1 -- 172.21.15.98:0/264589771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261415a620 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f2612ffd700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f260800f8b0 tx=0x7f2608009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608018070 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608006820 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261415a900 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608006be0 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.974+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614159060 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.975+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261415b100 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.975+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2608007030 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.975+0000 7f25fb7fe700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.975+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f26080842d0 con 0x7f26140a87c0 2023-10-06T01:40:54.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.975+0000 7f26137fe700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:40:54.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.976+0000 7f26137fe700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 0x7f26000693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2604000ef0 tx=0x7f2604015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:40:54.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:54.979+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f261415b100 con 0x7f26140a87c0 2023-10-06T01:40:55.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.205+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f261415b0c0 con 0x7f26140a87c0 2023-10-06T01:40:55.207 INFO:tasks.ceph.mon.a.smithi098.stderr:2023-10-06T01:40:55.206+0000 7f8a6565e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T01:40:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.210+0000 7f25fb7fe700 1 -- 172.21.15.98:0/264589771 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2608054d30 con 0x7f26140a87c0 2023-10-06T01:40:55.213 INFO:tasks.ceph.mds.d.smithi184.stderr: -3116> 2023-10-06T01:40:46.425+0000 7f8dca44b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 msgr2=0x7f26000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 0x7f26000693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2604000ef0 tx=0x7f2604015000 comp rx=0 tx=0).stop 2023-10-06T01:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 msgr2=0x7f261415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f260800f8b0 tx=0x7f2608009040 comp rx=0 tx=0).stop 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 shutdown_connections 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140a8c90 0x7f26140a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25fc003a60 0x7f2614158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 --2- 172.21.15.98:0/264589771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26140a87c0 0x7f261415c200 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 >> 172.21.15.98:0/264589771 conn(0x7f2614099830 msgr2=0x7f26140aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 shutdown_connections 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:40:55.212+0000 7f2619ae8700 1 -- 172.21.15.98:0/264589771 wait complete. 2023-10-06T01:40:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T01:40:55.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:40:55.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:40:55.239 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-06T01:40:55.269 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2023-10-06T01:40:55.270 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.271 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:40:55.272 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on skipped message = False 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.273 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2023-10-06T01:40:55.274 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:40:55.275 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 20 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 20 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 10 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2023-10-06T01:40:55.276 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2023-10-06T01:40:55.277 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.278 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.279 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: mon op complaint time = 120 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 600 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2023-10-06T01:40:55.280 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout:[mds] 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds = 20 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.281 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug frag = True 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug scatterstat = True 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: mds op complaint time = 180 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: mds verify scatter = False 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.282 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.283 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2023-10-06T01:40:55.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.285 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2023-10-06T01:40:55.286 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.287 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2023-10-06T01:40:55.288 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2023-10-06T01:40:55.288 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.288 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:40:55.288 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.288 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 20 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 20 2023-10-06T01:40:55.289 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 10 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2023-10-06T01:40:55.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2023-10-06T01:40:55.291 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.292 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2023-10-06T01:40:55.293 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2023-10-06T01:40:55.294 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = False 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:40:55.295 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.296 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.297 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:40:55.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T01:40:55.378 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2023-10-06T01:40:55.379 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2023-10-06T01:40:55.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2023-10-06T01:40:55.381 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on skipped message = False 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:40:55.382 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.383 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2023-10-06T01:40:55.383 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2023-10-06T01:40:55.383 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.383 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2023-10-06T01:40:55.383 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2023-10-06T01:40:55.385 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:40:55.385 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.385 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2023-10-06T01:40:55.385 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2023-10-06T01:40:55.385 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.386 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 20 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 20 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.387 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 10 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2023-10-06T01:40:55.388 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2023-10-06T01:40:55.389 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:40:55.390 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: mon op complaint time = 120 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2023-10-06T01:40:55.391 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 600 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2023-10-06T01:40:55.392 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout:[mds] 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds = 20 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug frag = True 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug scatterstat = False 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: mds op complaint time = 180 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: mds verify scatter = False 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.393 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.395 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2023-10-06T01:40:55.395 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2023-10-06T01:40:55.395 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T01:40:55.395 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2023-10-06T01:40:55.395 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2023-10-06T01:40:55.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2023-10-06T01:40:55.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2023-10-06T01:40:55.398 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.98,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.399 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2023-10-06T01:40:55.400 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2023-10-06T01:40:55.401 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 20 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 20 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 10 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2023-10-06T01:40:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.403 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.404 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2023-10-06T01:40:55.405 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2023-10-06T01:40:55.406 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2023-10-06T01:40:55.407 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = False 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = False 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-06T01:40:55.408 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-06T01:40:55.408 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T01:40:55.409 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T01:40:55.409 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T01:40:55.409 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:55.409 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T01:40:55.409 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T01:40:55.409 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T01:40:55.409 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:55.410 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T01:40:55.410 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T01:40:55.410 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T01:40:55.410 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:55.410 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T01:40:55.410 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T01:40:55.410 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T01:40:55.411 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:40:55.445 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T01:40:55.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T01:40:55.447 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T01:40:55.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T01:40:55.448 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T01:40:55.448 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T01:40:55.488 INFO:tasks.ceph.mds.c:Started 2023-10-06T01:40:55.489 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:55.490 INFO:tasks.ceph.mds.b:Started 2023-10-06T01:40:55.490 INFO:tasks.ceph.mds.d:Started 2023-10-06T01:40:55.491 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:55.491 DEBUG:teuthology.parallel:result is None 2023-10-06T01:40:55.558 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2023-10-06T01:40:55.561 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-06T01:40:55.561 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-06T01:41:01.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T01:41:01.445 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T01:41:01.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T01:41:01.448 INFO:tasks.ceph.mds.a:Started 2023-10-06T01:41:01.449 DEBUG:teuthology.parallel:result is None 2023-10-06T01:41:01.449 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T01:41:01.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 Processor -- start 2023-10-06T01:41:01.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 -- start start 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badeb6df00 0x55badeb26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 0x55badeb25cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badea6f620 con 0x55badeb510c0 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badea66400 con 0x55badeb6df00 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa20b2be000 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55badebf4830 con 0x55badebf7e10 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.488+0000 7fa2044f9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55314/0 (socket says 172.21.15.98:55314) 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 -- 172.21.15.98:0/1903665156 learned_addr learned my addr 172.21.15.98:0/1903665156 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa203cf8700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badeb6df00 0x55badeb26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa204cfa700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 0x55badeb25cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 -- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 msgr2=0x55badeb25cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 0x55badeb25cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 -- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badeb6df00 msgr2=0x55badeb26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badeb6df00 0x55badeb26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 -- 172.21.15.98:0/1903665156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8005db0 con 0x55badeb510c0 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa204cfa700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 0x55badeb25cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa2044f9700 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8005cf0 tx=0x7fa1f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99a54221fadcbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa202cf6700 1 -- 172.21.15.98:0/1903665156 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8005510 con 0x55badeb510c0 2023-10-06T01:41:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa202cf6700 1 -- 172.21.15.98:0/1903665156 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8000bc0 con 0x55badeb510c0 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.489+0000 7fa202cf6700 1 -- 172.21.15.98:0/1903665156 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8008bc0 con 0x55badeb510c0 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 -- 172.21.15.98:0/1903665156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 msgr2=0x55badeb54210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8005cf0 tx=0x7fa1f8016000 comp rx=0 tx=0).stop 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 -- 172.21.15.98:0/1903665156 shutdown_connections 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badebf7e10 0x55badeb25cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb510c0 0x55badeb54210 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 --2- 172.21.15.98:0/1903665156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badeb6df00 0x55badeb26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 -- 172.21.15.98:0/1903665156 >> 172.21.15.98:0/1903665156 conn(0x55badeb255b0 msgr2=0x55badeb27990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 -- 172.21.15.98:0/1903665156 shutdown_connections 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 -- 172.21.15.98:0/1903665156 wait complete. 2023-10-06T01:41:01.494 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.490+0000 7fa20b2be000 1 Processor -- start 2023-10-06T01:41:01.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 -- start start 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badeb510c0 0x55badebe7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badebf7e10 0x55badebe87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badebec870 con 0x55badeb6df00 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badebecb10 con 0x55badebf7e10 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa20b2be000 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55badebecdb0 con 0x55badeb510c0 2023-10-06T01:41:01.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa203cf8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa203cf8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55320/0 (socket says 172.21.15.98:55320) 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.491+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 learned_addr learned my addr 172.21.15.98:0/465485030 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badeb510c0 0x55badebe7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badebf7e10 0x55badebe87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badeb510c0 msgr2=0x55badebe7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badeb510c0 0x55badebe7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badebf7e10 msgr2=0x55badebe87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55badebf7e10 0x55badebe87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55badea71750 con 0x55badeb6df00 2023-10-06T01:41:01.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55badeb510c0 0x55badebe7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4008ac0 tx=0x7fa1f400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa2014f3700 1 -- 172.21.15.98:0/465485030 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400f040 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa2014f3700 1 -- 172.21.15.98:0/465485030 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f40094e0 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55badea981c0 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa2014f3700 1 -- 172.21.15.98:0/465485030 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4010ac0 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.492+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55badea7e3d0 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.494+0000 7fa2014f3700 1 -- 172.21.15.98:0/465485030 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1f4015450 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.494+0000 7fa2014f3700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 0x7fa1f0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa2014f3700 1 -- 172.21.15.98:0/465485030 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa1f40842a0 con 0x55badeb6df00 2023-10-06T01:41:01.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 0x7fa1f0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.500 INFO:teuthology.orchestra.run.smithi098.stdout:Warning: using slow linear search 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55badebea4e0 0x55badec16a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:1 6.0 6:00000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x55badebea4e0 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55badebea4e0 0x55badec16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 0x7fa1f0067d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8003080 tx=0x7fa1f801e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.495+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55badebea4e0 0x55badec16a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.496+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(1 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc014420 con 0x55badebea4e0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.496+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:2 6.10 6:08000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x55badebea4e0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.496+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(2 [pgnls start_epoch 40 out=177b] v36'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fa1fc014420 con 0x55badebea4e0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.496+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 0x7fa1fc012340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.496+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:3 6.8 6:10000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.497+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 0x7fa1fc012340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.497+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 0x7fa1fc012340 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.497+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== osd_op_reply(3 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.497+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:4 6.18 6:18000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x55badebea4e0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 3 ==== osd_op_reply(4 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc01b040 con 0x55badebea4e0 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7fa1fc01b290 0x7fa1fc01b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:5 6.4 6:20000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc01b290 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7fa1fc01b290 0x7fa1fc01b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7fa1fc01b290 0x7fa1fc01b700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.498+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== osd_op_reply(5 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f401b070 con 0x7fa1fc01b290 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fa1f4007070 0x7fa1f4055870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:6 6.14 6:28000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f4007070 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fa1f4007070 0x7fa1f4055870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fa1f4007070 0x7fa1f4055870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 1 ==== osd_op_reply(6 [pgnls start_epoch 40 out=149b] v36'18 uv18 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fa1fc01b040 con 0x7fa1f4007070 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7fa1fc022040 0x7fa1fc021400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:7 6.c 6:30000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.499+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7fa1fc022040 0x7fa1fc021400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7fa1fc022040 0x7fa1fc021400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(7 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:8 6.1c 6:38000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== osd_op_reply(8 [pgnls start_epoch 40 out=73b] v35'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7fa1f8007030 0x7fa1f801d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.500+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:9 6.2 6:40000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8007030 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.501+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7fa1f8007030 0x7fa1f801d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.501+0000 7fa203cf8700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7fa1f8007030 0x7fa1f801d930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(9 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f401b070 con 0x7fa1f8007030 2023-10-06T01:41:01.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:10 6.12 6:48000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 3 ==== osd_op_reply(10 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa2044f9700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7fa1f8012920 0x7fa1f8032400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:11 6.a 6:50000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8012920 2023-10-06T01:41:01.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.502+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7fa1f8012920 0x7fa1f8032400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.503+0000 7fa204cfa700 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7fa1f8012920 0x7fa1f8032400 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.503+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== osd_op_reply(11 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.503+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:12 6.1a 6:58000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8007030 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.503+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(12 [pgnls start_epoch 40 out=49b] v35'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f401b070 con 0x7fa1f8007030 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.503+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:13 6.6 6:60000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(13 [pgnls start_epoch 40 out=77b] v36'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:14 6.16 6:68000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x55badebea4e0 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 4 ==== osd_op_reply(14 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:15 6.e 6:70000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8007030 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 3 ==== osd_op_reply(15 [pgnls start_epoch 40 out=73b] v35'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1f401b070 con 0x7fa1f8007030 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.504+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:16 6.1e 6:78000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8007030 2023-10-06T01:41:01.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.505+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 4 ==== osd_op_reply(16 [pgnls start_epoch 40 out=49b] v35'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f401b070 con 0x7fa1f8007030 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.505+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:17 6.1 6:80000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc01b290 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.505+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== osd_op_reply(17 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f4060040 con 0x7fa1fc01b290 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.505+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:18 6.11 6:88000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.505+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 3 ==== osd_op_reply(18 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.506+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:19 6.9 6:90000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x55badebea4e0 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.506+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 5 ==== osd_op_reply(19 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.506+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:20 6.19 6:98000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8012920 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.506+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== osd_op_reply(20 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.506+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:21 6.5 6:a0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 4 ==== osd_op_reply(21 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:22 6.15 6:a8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 5 ==== osd_op_reply(22 [pgnls start_epoch 40 out=97b] v35'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:23 6.d 6:b0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8012920 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 3 ==== osd_op_reply(23 [pgnls start_epoch 40 out=74b] v35'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.507+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:24 6.1d 6:b8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc01b290 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.1 v2:172.21.15.98:6808/120656 3 ==== osd_op_reply(24 [pgnls start_epoch 40 out=73b] v35'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1f4060040 con 0x7fa1fc01b290 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:25 6.3 6:c0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc014be0 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 6 ==== osd_op_reply(25 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:26 6.13 6:c8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 4 ==== osd_op_reply(26 [pgnls start_epoch 40 out=98b] v36'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:27 6.b 6:d0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.508+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 5 ==== osd_op_reply(27 [pgnls start_epoch 40 out=76b] v36'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:28 6.1b 6:d8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 6 ==== osd_op_reply(28 [pgnls start_epoch 40 out=49b] v35'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:29 6.7 6:e0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8012920 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 4 ==== osd_op_reply(29 [pgnls start_epoch 40 out=73b] v35'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:30 6.17 6:e8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f8007030 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.509+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 5 ==== osd_op_reply(30 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1f4060040 con 0x7fa1f8007030 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:31 6.f 6:f0000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1f4007070 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 2 ==== osd_op_reply(31 [pgnls start_epoch 40 out=73b] v35'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:32 6.1f 6:f8000000:%01:::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55badebee7c0 con 0x7fa1fc022040 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 7 ==== osd_op_reply(32 [pgnls start_epoch 40 out=101b] v36'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc022040 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:33 6.10 6:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badebe9800 con 0x55badebea4e0 2023-10-06T01:41:01.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:34 6.10 6:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badebea0d0 con 0x55badebea4e0 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:35 6.10 6:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec17470 con 0x55badebea4e0 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:36 6.10 6:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec17d10 con 0x55badebea4e0 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:37 6.10 6:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec185b0 con 0x55badebea4e0 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:38 6.14 6:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec18e50 con 0x7fa1f4007070 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:39 6.14 6:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec196f0 con 0x7fa1f4007070 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:40 6.14 6:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec19f90 con 0x7fa1f4007070 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:41 6.14 6:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1a830 con 0x7fa1f4007070 2023-10-06T01:41:01.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:42 6.1c 6:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1b0d0 con 0x7fa1fc014be0 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:43 6.6 6:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1bac0 con 0x7fa1fc022040 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:44 6.e 6:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1c4e0 con 0x7fa1f8007030 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:45 6.15 6:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1cd80 con 0x7fa1fc014be0 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:46 6.15 6:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1a830 con 0x7fa1fc014be0 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:47 6.d 6:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1dd00 con 0x7fa1f8012920 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.510+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:48 6.1d 6:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1e810 con 0x7fa1fc01b290 2023-10-06T01:41:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.512+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 5 ==== osd_op_reply(47 mds_snaptable [delete] v40'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:49 6.13 6:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1ec10 con 0x7fa1fc022040 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 7 ==== osd_op_reply(42 607.00000000 [delete] v40'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f8027680 con 0x7fa1fc014be0 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:50 6.13 6:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1efc0 con 0x7fa1fc022040 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 8 ==== osd_op_reply(45 604.00000000 [delete] v40'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc014be0 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:51 6.b 6:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1f370 con 0x7fa1fc022040 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.7 v2:172.21.15.184:6816/120876 9 ==== osd_op_reply(46 500.00000000 [delete] v40'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc014be0 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.513+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:52 6.7 6:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1f720 con 0x7fa1f8012920 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.514+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 6 ==== osd_op_reply(33 601.00000000 [delete] v40'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.514+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:53 6.f 6:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1f370 con 0x7fa1f4007070 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.4 v2:172.21.15.184:6824/120887 6 ==== osd_op_reply(44 608.00000000 [delete] v40'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f4060040 con 0x7fa1f8007030 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 8 ==== osd_op_reply(49 mds0_inotable [delete] v40'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:54 6.1f 6:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1d750 con 0x7fa1fc022040 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:55 6.1f 6:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e40) v8 -- 0x55badec1f860 con 0x7fa1fc022040 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa203cf8700 1 -- 172.21.15.98:0/465485030 <== osd.1 v2:172.21.15.98:6808/120656 4 ==== osd_op_reply(48 605.00000000 [delete] v40'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f4060040 con 0x7fa1fc01b290 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 9 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v40'3 uv3 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 7 ==== osd_op_reply(34 602.00000000 [delete] v40'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 8 ==== osd_op_reply(35 10000000000.00000000 [delete] v40'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 3 ==== osd_op_reply(38 1.00000000.inode [delete] v40'19 uv19 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.515+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 4 ==== osd_op_reply(39 200.00000000 [delete] v40'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.516+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 5 ==== osd_op_reply(40 200.00000001 [delete] v40'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.517+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 10 ==== osd_op_reply(50 100.00000000 [delete] v40'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.517+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 11 ==== osd_op_reply(51 mds0_sessionmap [delete] v40'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.517+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 6 ==== osd_op_reply(53 400.00000000 [delete] v40'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.5 v2:172.21.15.184:6802/120874 6 ==== osd_op_reply(52 609.00000000 [delete] v40'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f8012920 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.2 v2:172.21.15.98:6824/120679 7 ==== osd_op_reply(41 606.00000000 [delete] v40'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x7fa1f4007070 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 9 ==== osd_op_reply(36 600.00000000 [delete] v40'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 <== osd.0 v2:172.21.15.98:6800/120654 10 ==== osd_op_reply(37 603.00000000 [delete] v40'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1fc02c040 con 0x55badebea4e0 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 12 ==== osd_op_reply(54 100.00000000.inode [delete] v40'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa2044f9700 1 -- 172.21.15.98:0/465485030 <== osd.3 v2:172.21.15.98:6809/120655 13 ==== osd_op_reply(55 1.00000000 [delete] v40'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa1f804c040 con 0x7fa1fc022040 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stdout:Removed 23 objects 2023-10-06T01:41:01.523 INFO:teuthology.orchestra.run.smithi098.stdout:successfully purged pool cephfs_metadata 2023-10-06T01:41:01.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55badebea4e0 msgr2=0x55badec16a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55badebea4e0 0x55badec16a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7fa1fc01b290 msgr2=0x7fa1fc01b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7fa1fc01b290 0x7fa1fc01b700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fa1f4007070 msgr2=0x7fa1f4055870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fa1f4007070 0x7fa1f4055870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7fa1fc022040 msgr2=0x7fa1fc021400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7fa1fc022040 0x7fa1fc021400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 reap_dead start 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7fa1f8007030 msgr2=0x7fa1f801d930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7fa1f8007030 0x7fa1f801d930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7fa1f8012920 msgr2=0x7fa1f8032400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7fa1f8012920 0x7fa1f8032400 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 msgr2=0x7fa1fc012340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa204cfa700 1 -- 172.21.15.98:0/465485030 reap_dead start 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 0x7fa1fc012340 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 msgr2=0x7fa1f0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 0x7fa1f0067d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8003080 tx=0x7fa1f801e000 comp rx=0 tx=0).stop 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 msgr2=0x55badebe82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4008ac0 tx=0x7fa1f400c040 comp rx=0 tx=0).stop 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 shutdown_connections 2023-10-06T01:41:01.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55badeb6df00 0x55badebe82b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7fa1fc014be0 0x7fa1fc012340 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 --2- 172.21.15.98:0/465485030 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa1f0065890 0x7fa1f0067d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.518+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 >> 172.21.15.98:0/465485030 conn(0x55badeb255b0 msgr2=0x55badebf58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:01.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.519+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 shutdown_connections 2023-10-06T01:41:01.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.519+0000 7fa20b2be000 1 -- 172.21.15.98:0/465485030 wait complete. 2023-10-06T01:41:01.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:01.534 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2023-10-06T01:41:01.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.647+0000 7f168479a700 1 Processor -- start 2023-10-06T01:41:01.649 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.647+0000 7f168479a700 1 -- start start 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f167c067840 0x7f167c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 0x7f167c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 0x7f167c0c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c0cc880 con 0x7f167c067840 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c0c4060 con 0x7f167c0cad90 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f168479a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0ccb00 con 0x7f167c067c10 2023-10-06T01:41:01.652 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.648+0000 7f1682536700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f167c067840 0x7f167c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f167c067840 0x7f167c06eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55360/0 (socket says 172.21.15.98:55360) 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 -- 172.21.15.98:0/3372886181 learned_addr learned my addr 172.21.15.98:0/3372886181 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682d37700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 0x7f167c0c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 -- 172.21.15.98:0/3372886181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 msgr2=0x7f167c0c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 0x7f167c0c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 -- 172.21.15.98:0/3372886181 >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 msgr2=0x7f167c0c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1681d35700 1 --2- 172.21.15.98:0/3372886181 >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 0x7f167c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 --2- 172.21.15.98:0/3372886181 >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 0x7f167c0c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 -- 172.21.15.98:0/3372886181 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c0ccde0 con 0x7f167c067840 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682d37700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 0x7f167c0c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:01.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1681d35700 1 --2- 172.21.15.98:0/3372886181 >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 0x7f167c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1682536700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c06eed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f167c06e1b0 tx=0x7f1678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c1807c84e2bbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1681534700 1 -- 172.21.15.98:0/3372886181 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167800b860 con 0x7f167c067840 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1681534700 1 -- 172.21.15.98:0/3372886181 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167800b9c0 con 0x7f167c067840 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.649+0000 7f1681534700 1 -- 172.21.15.98:0/3372886181 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167800bd60 con 0x7f167c067840 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 -- 172.21.15.98:0/3372886181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 msgr2=0x7f167c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c06eed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f167c06e1b0 tx=0x7f1678009250 comp rx=0 tx=0).stop 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 -- 172.21.15.98:0/3372886181 shutdown_connections 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c067c10 0x7f167c0c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 --2- 172.21.15.98:0/3372886181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c06eed0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 --2- 172.21.15.98:0/3372886181 >> v2:172.21.15.184:3300/0 conn(0x7f167c0cad90 0x7f167c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 -- 172.21.15.98:0/3372886181 >> 172.21.15.98:0/3372886181 conn(0x7f167c0691c0 msgr2=0x7f167c0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:01.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 -- 172.21.15.98:0/3372886181 shutdown_connections 2023-10-06T01:41:01.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.650+0000 7f168479a700 1 -- 172.21.15.98:0/3372886181 wait complete. 2023-10-06T01:41:01.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.651+0000 7f168479a700 1 Processor -- start 2023-10-06T01:41:01.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.651+0000 7f168479a700 1 -- start start 2023-10-06T01:41:01.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f167c067c10 0x7f167c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 0x7f167c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0c5c60 con 0x7f167c067840 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0c55e0 con 0x7f167c067c10 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1563a0 con 0x7f167c0cad90 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55362/0 (socket says 172.21.15.98:55362) 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 -- 172.21.15.98:0/451017992 learned_addr learned my addr 172.21.15.98:0/451017992 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682d37700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 0x7f167c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1681d35700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f167c067c10 0x7f167c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 -- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 msgr2=0x7f167c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 0x7f167c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 -- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f167c067c10 msgr2=0x7f167c155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f167c067c10 0x7f167c155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 -- 172.21.15.98:0/451017992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780094f0 con 0x7f167c067840 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682d37700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 0x7f167c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f1682536700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f167c0713d0 tx=0x7f1678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678024440 con 0x7f167c067840 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16780245a0 con 0x7f167c067840 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c1560c0 con 0x7f167c067840 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678024960 con 0x7f167c067840 2023-10-06T01:41:01.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.652+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c0ca2f0 con 0x7f167c067840 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.653+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c064ad0 con 0x7f167c067840 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.654+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f167801a420 con 0x7f167c067840 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.654+0000 7f16727fc700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 0x7f1660069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.654+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f16780866a0 con 0x7f167c067840 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.654+0000 7f1681d35700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 0x7f1660069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.654+0000 7f1681d35700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 0x7f1660069410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f167c0ca7e0 tx=0x7f166c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.658+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1678053620 con 0x7f167c067840 2023-10-06T01:41:01.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.844+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f167c0abe20 con 0x7f167c067840 2023-10-06T01:41:01.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.845+0000 7f16727fc700 1 -- 172.21.15.98:0/451017992 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17298 (secure 0 0 0) 0x7f1678056c40 con 0x7f167c067840 2023-10-06T01:41:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:01.847 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":40,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:40:56.033456+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 msgr2=0x7f1660069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 0x7f1660069410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f167c0ca7e0 tx=0x7f166c008040 comp rx=0 tx=0).stop 2023-10-06T01:41:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 msgr2=0x7f167c0c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f167c0713d0 tx=0x7f1678017000 comp rx=0 tx=0).stop 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 shutdown_connections 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1660066f60 0x7f1660069410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f167c0cad90 0x7f167c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f167c067840 0x7f167c0c9db0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 --2- 172.21.15.98:0/451017992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f167c067c10 0x7f167c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 >> 172.21.15.98:0/451017992 conn(0x7f167c0691c0 msgr2=0x7f167c0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 shutdown_connections 2023-10-06T01:41:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.847+0000 7f168479a700 1 -- 172.21.15.98:0/451017992 wait complete. 2023-10-06T01:41:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:40:39.889076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:40:40.800463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T01:41:01.861 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:41:01.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.975+0000 7fad8db9d700 1 Processor -- start 2023-10-06T01:41:01.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.975+0000 7fad8db9d700 1 -- start start 2023-10-06T01:41:01.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad880b2660 0x7fad880cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fad880b2a30 0x7fad880cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880ceae0 0x7fad880cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad880cc9b0 con 0x7fad880b2660 2023-10-06T01:41:01.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad880cf4c0 con 0x7fad880b2a30 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880cbda0 con 0x7fad880ceae0 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8cb9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad880b2660 0x7fad880cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8cb9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad880b2660 0x7fad880cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55378/0 (socket says 172.21.15.98:55378) 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.976+0000 7fad8cb9b700 1 -- 172.21.15.98:0/59554214 learned_addr learned my addr 172.21.15.98:0/59554214 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 -- 172.21.15.98:0/59554214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880ceae0 msgr2=0x7fad880cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8d39c700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880ceae0 0x7fad880cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad7ffff700 1 --2- 172.21.15.98:0/59554214 >> v2:172.21.15.184:3300/0 conn(0x7fad880b2a30 0x7fad880cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880ceae0 0x7fad880cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 -- 172.21.15.98:0/59554214 >> v2:172.21.15.184:3300/0 conn(0x7fad880b2a30 msgr2=0x7fad880cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/59554214 >> v2:172.21.15.184:3300/0 conn(0x7fad880b2a30 0x7fad880cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 -- 172.21.15.98:0/59554214 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880cc080 con 0x7fad880b2660 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880b2660 0x7fad880cccf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad880b6310 tx=0x7fad84009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db8e14c12cdca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad7f7fe700 1 -- 172.21.15.98:0/59554214 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84005c10 con 0x7fad880b2660 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad7f7fe700 1 -- 172.21.15.98:0/59554214 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84005d70 con 0x7fad880b2660 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad7f7fe700 1 -- 172.21.15.98:0/59554214 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400b880 con 0x7fad880b2660 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8db9d700 1 -- 172.21.15.98:0/59554214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880b2660 msgr2=0x7fad880cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.977+0000 7fad8db9d700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880b2660 0x7fad880cccf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad880b6310 tx=0x7fad84009220 comp rx=0 tx=0).stop 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 -- 172.21.15.98:0/59554214 shutdown_connections 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880ceae0 0x7fad880cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 --2- 172.21.15.98:0/59554214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880b2660 0x7fad880cccf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 --2- 172.21.15.98:0/59554214 >> v2:172.21.15.184:3300/0 conn(0x7fad880b2a30 0x7fad880cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 -- 172.21.15.98:0/59554214 >> 172.21.15.98:0/59554214 conn(0x7fad88099560 msgr2=0x7fad8806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:01.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 -- 172.21.15.98:0/59554214 shutdown_connections 2023-10-06T01:41:01.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.978+0000 7fad8db9d700 1 -- 172.21.15.98:0/59554214 wait complete. 2023-10-06T01:41:01.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.979+0000 7fad8db9d700 1 Processor -- start 2023-10-06T01:41:01.983 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.979+0000 7fad8db9d700 1 -- start start 2023-10-06T01:41:01.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.979+0000 7fad8db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880b2660 0x7fad8815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.979+0000 7fad8db9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad880b2a30 0x7fad8809d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.985 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8815ce70 con 0x7fad880ceae0 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8815c260 con 0x7fad880b2a30 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8815c540 con 0x7fad880b2660 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55386/0 (socket says 172.21.15.98:55386) 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 -- 172.21.15.98:0/2747738287 learned_addr learned my addr 172.21.15.98:0/2747738287 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad7ffff700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad880b2a30 0x7fad8809d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880b2660 0x7fad8815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 -- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880b2660 msgr2=0x7fad8815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.986 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880b2660 0x7fad8815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 -- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad880b2a30 msgr2=0x7fad8809d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad880b2a30 0x7fad8809d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 -- 172.21.15.98:0/2747738287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84009480 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8d39c700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad8000ceb0 tx=0x7fad80007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8000ad90 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80016430 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8815c820 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800167d0 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.980+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8809ded0 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.981+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8815d020 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.982+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad80003020 con 0x7fad880ceae0 2023-10-06T01:41:01.987 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.982+0000 7fad7cff9700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 0x7fad740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:01.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.982+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fad80012070 con 0x7fad880ceae0 2023-10-06T01:41:01.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.982+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 0x7fad740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:01.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.983+0000 7fad8cb9b700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 0x7fad740693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad84009260 tx=0x7fad84016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:01.988 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:01.986+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad8815d020 con 0x7fad880ceae0 2023-10-06T01:41:02.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.172+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad8805f270 con 0x7fad880ceae0 2023-10-06T01:41:02.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.172+0000 7fad7cff9700 1 -- 172.21.15.98:0/2747738287 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad800525e0 con 0x7fad880ceae0 2023-10-06T01:41:02.174 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 msgr2=0x7fad740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 0x7fad740693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad84009260 tx=0x7fad84016000 comp rx=0 tx=0).stop 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 msgr2=0x7fad881469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad8000ceb0 tx=0x7fad80007870 comp rx=0 tx=0).stop 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 shutdown_connections 2023-10-06T01:41:02.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fad74066f30 0x7fad740693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad880ceae0 0x7fad881469a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad880b2660 0x7fad8815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 --2- 172.21.15.98:0/2747738287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad880b2a30 0x7fad8809d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 >> 172.21.15.98:0/2747738287 conn(0x7fad88099560 msgr2=0x7fad88158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 shutdown_connections 2023-10-06T01:41:02.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.174+0000 7fad8db9d700 1 -- 172.21.15.98:0/2747738287 wait complete. 2023-10-06T01:41:02.187 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:41:02.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.303+0000 7fd0de90d700 1 Processor -- start 2023-10-06T01:41:02.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.303+0000 7fd0de90d700 1 -- start start 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0de90d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0de90d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 0x7fd0d80cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0de90d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d80cf350 con 0x7fd0d80ce820 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0de90d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d80cf610 con 0x7fd0d8159510 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0de90d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d80cf8f0 con 0x7fd0d81598e0 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 0x7fd0d80cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:44204/0 (socket says 172.21.15.98:44204) 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d7fff700 1 -- 172.21.15.98:0/1720201942 learned_addr learned my addr 172.21.15.98:0/1720201942 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0dceaa700 1 --1- 172.21.15.98:0/1720201942 >> v1:172.21.15.184:6789/0 conn(0x7fd0d80ce820 0x7fd0d80cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:55410/0 (socket says 172.21.15.98:55410) 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 -- 172.21.15.98:0/1720201942 >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 msgr2=0x7fd0d80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 --2- 172.21.15.98:0/1720201942 >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 -- 172.21.15.98:0/1720201942 >> v1:172.21.15.184:6789/0 conn(0x7fd0d80ce820 legacy=0x7fd0d80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 -- 172.21.15.98:0/1720201942 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d80cfbd0 con 0x7fd0d81598e0 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d7fff700 1 --2- 172.21.15.98:0/1720201942 >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.304+0000 7fd0d77fe700 1 --2- 172.21.15.98:0/1720201942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 0x7fd0d80cbae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0098e0 tx=0x7fd0cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f152a2de8b3eb97d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0d6ffd700 1 -- 172.21.15.98:0/1720201942 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00b640 con 0x7fd0d81598e0 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0d6ffd700 1 -- 172.21.15.98:0/1720201942 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc00b7a0 con 0x7fd0d81598e0 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0d6ffd700 1 -- 172.21.15.98:0/1720201942 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00bb40 con 0x7fd0d81598e0 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 -- 172.21.15.98:0/1720201942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 msgr2=0x7fd0d80cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1720201942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 0x7fd0d80cbae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0098e0 tx=0x7fd0cc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 -- 172.21.15.98:0/1720201942 shutdown_connections 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1720201942 >> v2:172.21.15.184:3300/0 conn(0x7fd0d8159510 0x7fd0d80cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1720201942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d81598e0 0x7fd0d80cbae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 -- 172.21.15.98:0/1720201942 >> 172.21.15.98:0/1720201942 conn(0x7fd0d8089e90 msgr2=0x7fd0d80997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 -- 172.21.15.98:0/1720201942 shutdown_connections 2023-10-06T01:41:02.309 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.305+0000 7fd0de90d700 1 -- 172.21.15.98:0/1720201942 wait complete. 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.306+0000 7fd0de90d700 1 Processor -- start 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 -- start start 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d80ce820 0x7fd0d815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d8159510 0x7fd0d814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.312 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814d670 con 0x7fd0d81598e0 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d8139720 con 0x7fd0d80ce820 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0de90d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814e470 con 0x7fd0d8159510 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0dceaa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0dceaa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55390/0 (socket says 172.21.15.98:55390) 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0dceaa700 1 -- 172.21.15.98:0/1364690239 learned_addr learned my addr 172.21.15.98:0/1364690239 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0d7fff700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d80ce820 0x7fd0d815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.307+0000 7fd0d77fe700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d8159510 0x7fd0d814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 -- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d8159510 msgr2=0x7fd0d814d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d8159510 0x7fd0d814d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 -- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d80ce820 msgr2=0x7fd0d815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.313 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d80ce820 0x7fd0d815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 -- 172.21.15.98:0/1364690239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80094f0 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0dceaa700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000cd00 tx=0x7fd0d0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ce60 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0005df0 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc009420 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ab00 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.308+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d814db40 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.309+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d8064ad0 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.309+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0d00070f0 con 0x7fd0d81598e0 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.310+0000 7fd0bffff700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 0x7fd0c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.310+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fd0d00848c0 con 0x7fd0d81598e0 2023-10-06T01:41:02.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.310+0000 7fd0d7fff700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 0x7fd0c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.310+0000 7fd0d7fff700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 0x7fd0c40693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8006d20 tx=0x7fd0c8009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.314+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0d0051840 con 0x7fd0d81598e0 2023-10-06T01:41:02.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.498+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0d80d02c0 con 0x7fd0d81598e0 2023-10-06T01:41:02.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.499+0000 7fd0bffff700 1 -- 172.21.15.98:0/1364690239 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0d0054e60 con 0x7fd0d81598e0 2023-10-06T01:41:02.501 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:41:02.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.500+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 msgr2=0x7fd0c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.500+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 0x7fd0c40693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8006d20 tx=0x7fd0c8009250 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.500+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 msgr2=0x7fd0d813bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.500+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000cd00 tx=0x7fd0d0001b10 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 shutdown_connections 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd0c4066f30 0x7fd0c40693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0d81598e0 0x7fd0d813bf80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d80ce820 0x7fd0d815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 --2- 172.21.15.98:0/1364690239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d8159510 0x7fd0d814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 >> 172.21.15.98:0/1364690239 conn(0x7fd0d8089e90 msgr2=0x7fd0d80c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 shutdown_connections 2023-10-06T01:41:02.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.501+0000 7fd0de90d700 1 -- 172.21.15.98:0/1364690239 wait complete. 2023-10-06T01:41:02.514 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:41:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.629+0000 7f9d1e174700 1 Processor -- start 2023-10-06T01:41:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.629+0000 7f9d1e174700 1 -- start start 2023-10-06T01:41:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d1e174700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 0x7f9d180cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d1e174700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d180cbae0 0x7f9d180cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d1e174700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180cbf20 con 0x7f9d181598e0 2023-10-06T01:41:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d1e174700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180cf610 con 0x7f9d180cbae0 2023-10-06T01:41:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d1e174700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180cf8f0 con 0x7f9d18159510 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d16ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9d181598e0 0x7f9d180ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55258/0 (socket says 172.21.15.98:55258) 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d17fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d180cbae0 0x7f9d180cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.630+0000 7f9d16ffd700 1 -- 172.21.15.98:0/111556182 learned_addr learned my addr 172.21.15.98:0/111556182 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d177fe700 1 --2- 172.21.15.98:0/111556182 >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 0x7f9d180cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d17fff700 1 -- 172.21.15.98:0/111556182 >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 msgr2=0x7f9d180cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d17fff700 1 --2- 172.21.15.98:0/111556182 >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 0x7f9d180cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d17fff700 1 -- 172.21.15.98:0/111556182 >> v1:172.21.15.98:6789/0 conn(0x7f9d181598e0 legacy=0x7f9d180ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d17fff700 1 -- 172.21.15.98:0/111556182 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180cfbd0 con 0x7f9d180cbae0 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d167fc700 1 -- 172.21.15.98:0/111556182 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720990777 0 0) 0x7f9d180cbf20 con 0x7f9d181598e0 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d177fe700 1 --2- 172.21.15.98:0/111556182 >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 0x7f9d180cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d17fff700 1 --2- 172.21.15.98:0/111556182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d180cbae0 0x7f9d180cef30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d18088fb0 tx=0x7f9d100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f374069e3ef0d12d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d167fc700 1 -- 172.21.15.98:0/111556182 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000b6b0 con 0x7f9d180cbae0 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d167fc700 1 -- 172.21.15.98:0/111556182 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000b810 con 0x7f9d180cbae0 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d167fc700 1 -- 172.21.15.98:0/111556182 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000bbb0 con 0x7f9d180cbae0 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 -- 172.21.15.98:0/111556182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d180cbae0 msgr2=0x7f9d180cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 --2- 172.21.15.98:0/111556182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d180cbae0 0x7f9d180cef30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d18088fb0 tx=0x7f9d100091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 -- 172.21.15.98:0/111556182 shutdown_connections 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 --2- 172.21.15.98:0/111556182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d180cbae0 0x7f9d180cef30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 --2- 172.21.15.98:0/111556182 >> v2:172.21.15.184:3300/0 conn(0x7f9d18159510 0x7f9d180cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 -- 172.21.15.98:0/111556182 >> 172.21.15.98:0/111556182 conn(0x7f9d18089e90 msgr2=0x7f9d180997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 -- 172.21.15.98:0/111556182 shutdown_connections 2023-10-06T01:41:02.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.631+0000 7f9d1e174700 1 -- 172.21.15.98:0/111556182 wait complete. 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 Processor -- start 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 -- start start 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 0x7f9d18156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 0x7f9d1816cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815ac60 con 0x7f9d18159510 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1816d2d0 con 0x7f9d180cbae0 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.633+0000 7f9d1e174700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180b4d90 con 0x7f9d181598e0 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54920/0 (socket says 172.21.15.98:54920) 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 -- 172.21.15.98:0/139438607 learned_addr learned my addr 172.21.15.98:0/139438607 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d177fe700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 0x7f9d18156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d17fff700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 0x7f9d1816cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 -- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 msgr2=0x7f9d1816cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 0x7f9d1816cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 -- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 msgr2=0x7f9d18156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 0x7f9d18156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 -- 172.21.15.98:0/139438607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009420 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d177fe700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 0x7f9d18156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d17fff700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 0x7f9d1816cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d16ffd700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c510 tx=0x7f9d0c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c005b30 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c005c90 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180b4460 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c007300 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.634+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180b4a20 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.635+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d18089540 con 0x7f9d18159510 2023-10-06T01:41:02.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.635+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d0c01b3b0 con 0x7f9d18159510 2023-10-06T01:41:02.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.636+0000 7f9cff7fe700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.636+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9d0c083d40 con 0x7f9d18159510 2023-10-06T01:41:02.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.636+0000 7f9d177fe700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.636+0000 7f9d177fe700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 0x7f9d040693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800d640 tx=0x7f9d08007670 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.640+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d0c050cc0 con 0x7f9d18159510 2023-10-06T01:41:02.825 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.824+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d180b4f00 con 0x7f9d18159510 2023-10-06T01:41:02.826 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.824+0000 7f9cff7fe700 1 -- 172.21.15.98:0/139438607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d0c0170a0 con 0x7f9d18159510 2023-10-06T01:41:02.826 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:41:02.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 msgr2=0x7f9d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 0x7f9d040693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800d640 tx=0x7f9d08007670 comp rx=0 tx=0).stop 2023-10-06T01:41:02.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 msgr2=0x7f9d1815a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c510 tx=0x7f9d0c00c8d0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 shutdown_connections 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d181598e0 0x7f9d1816cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d180cbae0 0x7f9d18156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 --2- 172.21.15.98:0/139438607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d18159510 0x7f9d1815a720 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 >> 172.21.15.98:0/139438607 conn(0x7f9d18089e90 msgr2=0x7f9d1814b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 shutdown_connections 2023-10-06T01:41:02.829 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.826+0000 7f9d1e174700 1 -- 172.21.15.98:0/139438607 wait complete. 2023-10-06T01:41:02.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:41:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.954+0000 7fe0b72ab700 1 Processor -- start 2023-10-06T01:41:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.954+0000 7fe0b72ab700 1 -- start start 2023-10-06T01:41:02.964 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b72ab700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0b00b0a50 0x7fe0b00cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.964 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b72ab700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00c5400 con 0x7fe0b00cc460 2023-10-06T01:41:02.964 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b72ab700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00ccd40 con 0x7fe0b00b0680 2023-10-06T01:41:02.964 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b72ab700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00ca7d0 con 0x7fe0b00b0a50 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b5848700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe0b00cc460 0x7fe0b00c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55262/0 (socket says 172.21.15.98:55262) 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.955+0000 7fe0b5848700 1 -- 172.21.15.98:0/3716376117 learned_addr learned my addr 172.21.15.98:0/3716376117 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b4846700 1 --2- 172.21.15.98:0/3716376117 >> v2:172.21.15.184:3300/0 conn(0x7fe0b00b0a50 0x7fe0b00cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568277479 0 0) 0x7fe0b00c5400 con 0x7fe0b00cc460 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe098003570 con 0x7fe0b00cc460 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449388521 0 0) 0x7fe098003570 con 0x7fe0b00cc460 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 >> v2:172.21.15.184:3300/0 conn(0x7fe0b00b0a50 msgr2=0x7fe0b00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 --2- 172.21.15.98:0/3716376117 >> v2:172.21.15.184:3300/0 conn(0x7fe0b00b0a50 0x7fe0b00cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 >> v1:172.21.15.184:6789/0 conn(0x7fe0b00b0680 legacy=0x7fe0b00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00cd020 con 0x7fe0b00cc460 2023-10-06T01:41:02.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fe0a8001470 con 0x7fe0b00cc460 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 >> v1:172.21.15.98:6789/0 conn(0x7fe0b00cc460 legacy=0x7fe0b00c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 0x7fe098003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe098004810 0x7fe098004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0980051c0 0x7fe09800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00ca7d0 con 0x7fe098003a60 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09800c0e0 con 0x7fe0980051c0 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09800c2a0 con 0x7fe098004810 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 0x7fe098003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5848700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0980051c0 0x7fe09800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b4846700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe098004810 0x7fe098004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 -- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe098004810 msgr2=0x7fe098004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:02.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe098004810 0x7fe098004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 -- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0980051c0 msgr2=0x7fe09800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0980051c0 0x7fe09800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 -- 172.21.15.98:0/3716376117 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098003570 con 0x7fe098003a60 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5848700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0980051c0 0x7fe09800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5848700 1 -- 172.21.15.98:0/3716376117 reap_dead start 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.956+0000 7fe0b5047700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 0x7fe098003e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c3e0 tx=0x7fe0a0001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b52676d10bf0a2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0001f10 con 0x7fe098003a60 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a000ccd0 con 0x7fe098003a60 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0affff700 1 -- 172.21.15.98:0/3716376117 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0012650 con 0x7fe098003a60 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 -- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 msgr2=0x7fe098003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 0x7fe098003e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c3e0 tx=0x7fe0a0001c00 comp rx=0 tx=0).stop 2023-10-06T01:41:02.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 -- 172.21.15.98:0/3716376117 shutdown_connections 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/3716376117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098003a60 0x7fe098003e30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 -- 172.21.15.98:0/3716376117 >> 172.21.15.98:0/3716376117 conn(0x7fe0b0088000 msgr2=0x7fe0b00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 -- 172.21.15.98:0/3716376117 shutdown_connections 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.957+0000 7fe0b72ab700 1 -- 172.21.15.98:0/3716376117 wait complete. 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.958+0000 7fe0b72ab700 1 Processor -- start 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.958+0000 7fe0b72ab700 1 -- start start 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 0x7fe0b0155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b0159360 0x7fe0b01597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0167d80 con 0x7fe0b00c65c0 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0155970 con 0x7fe098003a60 2023-10-06T01:41:02.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b72ab700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00c6f70 con 0x7fe0b0159360 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b4846700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b4846700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54938/0 (socket says 172.21.15.98:54938) 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b4846700 1 -- 172.21.15.98:0/4234951484 learned_addr learned my addr 172.21.15.98:0/4234951484 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b5047700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 0x7fe0b0155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.959+0000 7fe0b5848700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b0159360 0x7fe0b01597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 -- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b0159360 msgr2=0x7fe0b01597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b0159360 0x7fe0b01597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 -- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 msgr2=0x7fe0b0155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 0x7fe0b0155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 -- 172.21.15.98:0/4234951484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b009a8c0 con 0x7fe0b00c65c0 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b5047700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 0x7fe0b0155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:02.969 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b4846700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4011990 tx=0x7fe0a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400f6d0 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a400f830 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400fbf0 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b015a050 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.960+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b015a610 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.961+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b009aa50 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.961+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0a4015460 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.962+0000 7fe0ad7fa700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 0x7fe09c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.962+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fe0a4085030 con 0x7fe0b00c65c0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.962+0000 7fe0b5047700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 0x7fe09c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.962+0000 7fe0b5047700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 0x7fe09c0693b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00013a0 tx=0x7fe0a0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:02.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:02.966+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0a4051fb0 con 0x7fe0b00c65c0 2023-10-06T01:41:03.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.150+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0b0064ad0 con 0x7fe0b00c65c0 2023-10-06T01:41:03.152 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.150+0000 7fe0ad7fa700 1 -- 172.21.15.98:0/4234951484 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0a40555d0 con 0x7fe0b00c65c0 2023-10-06T01:41:03.152 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:41:03.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 msgr2=0x7fe09c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:03.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 0x7fe09c0693b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00013a0 tx=0x7fe0a0016000 comp rx=0 tx=0).stop 2023-10-06T01:41:03.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 msgr2=0x7fe0b00c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:03.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4011990 tx=0x7fe0a4008040 comp rx=0 tx=0).stop 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 shutdown_connections 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe09c066f00 0x7fe09c0693b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b0159360 0x7fe0b01597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe098003a60 0x7fe0b0155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 --2- 172.21.15.98:0/4234951484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0b00c65c0 0x7fe0b00c6a30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 >> 172.21.15.98:0/4234951484 conn(0x7fe0b0088000 msgr2=0x7fe0b00c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 shutdown_connections 2023-10-06T01:41:03.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.152+0000 7fe0b72ab700 1 -- 172.21.15.98:0/4234951484 wait complete. 2023-10-06T01:41:03.165 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'recovery_fs' 2023-10-06T01:41:03.165 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create recovery_fs_metadata --pg_num_min 64 2023-10-06T01:41:03.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.282+0000 7f7d38bb0700 1 Processor -- start 2023-10-06T01:41:03.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.282+0000 7f7d38bb0700 1 -- start start 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7d3406ed20 0x7f7d340b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d3406f0f0 0x7f7d340b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d340af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3406fb80 con 0x7f7d3406f0f0 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340b2d20 con 0x7f7d3406ed20 2023-10-06T01:41:03.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d38bb0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340b0450 con 0x7f7d340aee20 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d3406f0f0 0x7f7d340b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d3406f0f0 0x7f7d340b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54942/0 (socket says 172.21.15.98:54942) 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 -- 172.21.15.98:0/3111936650 learned_addr learned my addr 172.21.15.98:0/3111936650 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d3259c700 1 --2- 172.21.15.98:0/3111936650 >> v2:172.21.15.184:3300/0 conn(0x7f7d3406ed20 0x7f7d340b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d32d9d700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d340af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 -- 172.21.15.98:0/3111936650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 msgr2=0x7f7d340af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d340af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 -- 172.21.15.98:0/3111936650 >> v2:172.21.15.184:3300/0 conn(0x7f7d3406ed20 msgr2=0x7f7d340b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3111936650 >> v2:172.21.15.184:3300/0 conn(0x7f7d3406ed20 0x7f7d340b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 -- 172.21.15.98:0/3111936650 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340af7c0 con 0x7f7d3406f0f0 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.283+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406f0f0 0x7f7d340b08e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cdd30 tx=0x7f7d1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addf8fcaa936ef93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:03.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d31d9b700 1 -- 172.21.15.98:0/3111936650 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00b860 con 0x7f7d3406f0f0 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d31d9b700 1 -- 172.21.15.98:0/3111936650 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00b9c0 con 0x7f7d3406f0f0 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d31d9b700 1 -- 172.21.15.98:0/3111936650 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00bd60 con 0x7f7d3406f0f0 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3111936650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406f0f0 msgr2=0x7f7d340b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406f0f0 0x7f7d340b08e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cdd30 tx=0x7f7d1c009250 comp rx=0 tx=0).stop 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3111936650 shutdown_connections 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3111936650 >> v2:172.21.15.184:3300/0 conn(0x7f7d3406ed20 0x7f7d340b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d340af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3111936650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406f0f0 0x7f7d340b08e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3111936650 >> 172.21.15.98:0/3111936650 conn(0x7f7d34068330 msgr2=0x7f7d34068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3111936650 shutdown_connections 2023-10-06T01:41:03.288 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.284+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3111936650 wait complete. 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.285+0000 7f7d38bb0700 1 Processor -- start 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.285+0000 7f7d38bb0700 1 -- start start 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3406f0f0 0x7f7d34159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d34159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34158700 con 0x7f7d3406ed20 2023-10-06T01:41:03.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d341589a0 con 0x7f7d3406f0f0 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d38bb0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34158c80 con 0x7f7d340aee20 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54944/0 (socket says 172.21.15.98:54944) 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 -- 172.21.15.98:0/3880430762 learned_addr learned my addr 172.21.15.98:0/3880430762 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d32d9d700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d34159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3406f0f0 0x7f7d34159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 -- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 msgr2=0x7f7d34159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d34159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 -- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3406f0f0 msgr2=0x7f7d34159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3406f0f0 0x7f7d34159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.286+0000 7f7d3259c700 1 -- 172.21.15.98:0/3880430762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0094f0 con 0x7f7d3406ed20 2023-10-06T01:41:03.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d3259c700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400d860 tx=0x7f7d24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240125f0 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24012750 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d34158f60 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24012b10 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.287+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3416a8b0 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.288+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d341592d0 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.288+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d2401e460 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.289+0000 7f7d2affd700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 0x7f7d14069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.289+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f7d24084da0 con 0x7f7d3406ed20 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.289+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 0x7f7d14069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:03.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.289+0000 7f7d29d9b700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 0x7f7d14069460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cdd30 tx=0x7f7d1c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:03.294 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.293+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d24055730 con 0x7f7d3406ed20 2023-10-06T01:41:03.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:03.475+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7d34064ad0 con 0x7f7d3406ed20 2023-10-06T01:41:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.056+0000 7f7d2affd700 1 -- 172.21.15.98:0/3880430762 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64}]=0 pool 'recovery_fs_metadata' created v42) v1 ==== 148+0+0 (secure 0 0 0) 0x7f7d240552c0 con 0x7f7d3406ed20 2023-10-06T01:41:04.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.060+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 msgr2=0x7f7d14069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.060+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 0x7f7d14069460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cdd30 tx=0x7f7d1c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:04.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.060+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 msgr2=0x7f7d3415c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.060+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400d860 tx=0x7f7d24008040 comp rx=0 tx=0).stop 2023-10-06T01:41:04.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 shutdown_connections 2023-10-06T01:41:04.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d14066fb0 0x7f7d14069460 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d340aee20 0x7f7d34159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d3406ed20 0x7f7d3415c5c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 --2- 172.21.15.98:0/3880430762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3406f0f0 0x7f7d34159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.063 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.061+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 >> 172.21.15.98:0/3880430762 conn(0x7f7d34068330 msgr2=0x7f7d340ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:04.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.062+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 shutdown_connections 2023-10-06T01:41:04.064 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.063+0000 7f7d38bb0700 1 -- 172.21.15.98:0/3880430762 wait complete. 2023-10-06T01:41:04.066 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'recovery_fs_metadata' created 2023-10-06T01:41:04.077 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T01:41:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.197+0000 7f25623ac700 1 Processor -- start 2023-10-06T01:41:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.197+0000 7f25623ac700 1 -- start start 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f25623ac700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f255c140fa0 0x7f255c0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f25623ac700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c15bda0 con 0x7f255c140bd0 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f25623ac700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c0c78c0 con 0x7f255c15b920 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f25623ac700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c06ada0 con 0x7f255c140fa0 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f255bfff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f255c140bd0 0x7f255c06a690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55270/0 (socket says 172.21.15.98:55270) 2023-10-06T01:41:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.198+0000 7f255bfff700 1 -- 172.21.15.98:0/1345771321 learned_addr learned my addr 172.21.15.98:0/1345771321 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255b7fe700 1 --2- 172.21.15.98:0/1345771321 >> v2:172.21.15.184:3300/0 conn(0x7f255c140fa0 0x7f255c0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012073633 0 0) 0x7f255c15bda0 con 0x7f255c140bd0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2544003570 con 0x7f255c140bd0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198279223 0 0) 0x7f2544003570 con 0x7f255c140bd0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 >> v2:172.21.15.184:3300/0 conn(0x7f255c140fa0 msgr2=0x7f255c0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 --2- 172.21.15.98:0/1345771321 >> v2:172.21.15.184:3300/0 conn(0x7f255c140fa0 0x7f255c0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 >> v1:172.21.15.184:6789/0 conn(0x7f255c15b920 legacy=0x7f255c0675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0c7ba0 con 0x7f255c140bd0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f254c001470 con 0x7f255c140bd0 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 >> v1:172.21.15.98:6789/0 conn(0x7f255c140bd0 legacy=0x7f255c06a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2544003a60 0x7f2544003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2544004810 0x7f2544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f254400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c06ada0 con 0x7f25440051c0 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254400c050 con 0x7f2544003a60 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254400c210 con 0x7f2544004810 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f254400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255bfff700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2544003a60 0x7f2544003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255b7fe700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2544004810 0x7f2544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 -- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2544004810 msgr2=0x7f2544004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2544004810 0x7f2544004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 -- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2544003a60 msgr2=0x7f2544003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2544003a60 0x7f2544003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 -- 172.21.15.98:0/1345771321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544003570 con 0x7f25440051c0 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f255bfff700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2544003a60 0x7f2544003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 -- 172.21.15.98:0/1345771321 reap_dead start 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.199+0000 7f2560949700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f254400ba60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f25540020a0 tx=0x7f25540016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3242297e6413b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25540019e0 con 0x7f25440051c0 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400c8b0 con 0x7f25440051c0 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f255affd700 1 -- 172.21.15.98:0/1345771321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400cc70 con 0x7f25440051c0 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 -- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 msgr2=0x7f254400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f254400ba60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f25540020a0 tx=0x7f25540016d0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 -- 172.21.15.98:0/1345771321 shutdown_connections 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 --2- 172.21.15.98:0/1345771321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f254400ba60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 -- 172.21.15.98:0/1345771321 >> 172.21.15.98:0/1345771321 conn(0x7f255c0cd1e0 msgr2=0x7f255c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:04.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 -- 172.21.15.98:0/1345771321 shutdown_connections 2023-10-06T01:41:04.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.200+0000 7f25623ac700 1 -- 172.21.15.98:0/1345771321 wait complete. 2023-10-06T01:41:04.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.201+0000 7f25623ac700 1 Processor -- start 2023-10-06T01:41:04.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.201+0000 7f25623ac700 1 -- start start 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 0x7f255c157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 0x7f255c0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c158750 con 0x7f25440051c0 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c5020 con 0x7f255c1582d0 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f25623ac700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c4410 con 0x7f255c0aefe0 2023-10-06T01:41:04.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54950/0 (socket says 172.21.15.98:54950) 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 -- 172.21.15.98:0/3971631777 learned_addr learned my addr 172.21.15.98:0/3971631777 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255b7fe700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 0x7f255c157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f2560949700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 0x7f255c0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 -- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 msgr2=0x7f255c157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 0x7f255c157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 -- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 msgr2=0x7f255c0c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 0x7f255c0c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255bfff700 1 -- 172.21.15.98:0/3971631777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0c46f0 con 0x7f25440051c0 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f255b7fe700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 0x7f255c157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.202+0000 7f2560949700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 0x7f255c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:04.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f255bfff700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f254c00eb00 tx=0x7f254c000ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c005e10 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c010040 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c0c49d0 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00a9b0 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.203+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0af450 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.204+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c064ad0 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.204+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f254c00ad20 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.205+0000 7f2543fff700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 0x7f25480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.205+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6839+0+0 (secure 0 0 0) 0x7f254c0844d0 con 0x7f25440051c0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.205+0000 7f255b7fe700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 0x7f25480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.205+0000 7f255b7fe700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 0x7f25480693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2550006f00 tx=0x7f2550008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.208+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f254c051400 con 0x7f25440051c0 2023-10-06T01:41:04.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.392+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f255c0c7e80 con 0x7f25440051c0 2023-10-06T01:41:04.394 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.392+0000 7f2543fff700 1 -- 172.21.15.98:0/3971631777 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' already exists v42) v1 ==== 177+0+0 (secure 0 0 0) 0x7f254c0190d0 con 0x7f25440051c0 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 msgr2=0x7f25480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 0x7f25480693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2550006f00 tx=0x7f2550008040 comp rx=0 tx=0).stop 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 msgr2=0x7f255c0c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f254c00eb00 tx=0x7f254c000ca0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 shutdown_connections 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2548066f30 0x7f25480693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f255c1582d0 0x7f255c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25440051c0 0x7f255c0c54b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 --2- 172.21.15.98:0/3971631777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f255c0aefe0 0x7f255c157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 >> 172.21.15.98:0/3971631777 conn(0x7f255c0cd1e0 msgr2=0x7f255c0c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:04.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 shutdown_connections 2023-10-06T01:41:04.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.394+0000 7f25623ac700 1 -- 172.21.15.98:0/3971631777 wait complete. 2023-10-06T01:41:04.397 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' already exists 2023-10-06T01:41:04.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new recovery_fs recovery_fs_metadata cephfs_data --recover --allow-dangerous-metadata-overlay 2023-10-06T01:41:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.524+0000 7f496a9c2700 1 Processor -- start 2023-10-06T01:41:04.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.524+0000 7f496a9c2700 1 -- start start 2023-10-06T01:41:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f496a9c2700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49640b23a0 0x7f49640b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f496a9c2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f496a9c2700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49640b2d20 con 0x7f496406ed20 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f496a9c2700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496406fb80 con 0x7f49640b23a0 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f496a9c2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640b2840 con 0x7f49640aee20 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f49637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49640b23a0 0x7f49640b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f4968f5f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f49637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49640b23a0 0x7f49640b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:54938/0 (socket says 172.21.15.98:54938) 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f4963fff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f496406ed20 0x7f496406f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:40264/0 (socket says 172.21.15.98:40264) 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f4968f5f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:55092/0 (socket says 172.21.15.98:55092) 2023-10-06T01:41:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.525+0000 7f49637fe700 1 -- 172.21.15.98:0/4266231806 learned_addr learned my addr 172.21.15.98:0/4266231806 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f49637fe700 1 -- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 msgr2=0x7f49640af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f49637fe700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f49637fe700 1 -- 172.21.15.98:0/4266231806 >> v1:172.21.15.184:6789/0 conn(0x7f496406ed20 legacy=0x7f496406f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f49637fe700 1 -- 172.21.15.98:0/4266231806 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640b0450 con 0x7f49640b23a0 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f4962ffd700 1 -- 172.21.15.98:0/4266231806 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500761612 0 0) 0x7f49640b2d20 con 0x7f496406ed20 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f4968f5f700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f49637fe700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49640b23a0 0x7f49640b08e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49580099e0 tx=0x7f4958009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33466c29dbf0753 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f4962ffd700 1 -- 172.21.15.98:0/4266231806 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800b820 con 0x7f49640b23a0 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f4962ffd700 1 -- 172.21.15.98:0/4266231806 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800b980 con 0x7f49640b23a0 2023-10-06T01:41:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f4962ffd700 1 -- 172.21.15.98:0/4266231806 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800bd20 con 0x7f49640b23a0 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f496a9c2700 1 -- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49640b23a0 msgr2=0x7f49640b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.526+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49640b23a0 0x7f49640b08e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49580099e0 tx=0x7f4958009250 comp rx=0 tx=0).stop 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 -- 172.21.15.98:0/4266231806 shutdown_connections 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640aee20 0x7f49640af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4266231806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49640b23a0 0x7f49640b08e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 -- 172.21.15.98:0/4266231806 >> 172.21.15.98:0/4266231806 conn(0x7f4964068330 msgr2=0x7f4964068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 -- 172.21.15.98:0/4266231806 shutdown_connections 2023-10-06T01:41:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.527+0000 7f496a9c2700 1 -- 172.21.15.98:0/4266231806 wait complete. 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.528+0000 7f496a9c2700 1 Processor -- start 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.528+0000 7f496a9c2700 1 -- start start 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f496406ed20 0x7f496415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640b23a0 0x7f49640ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4964159910 con 0x7f49640aee20 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496416e190 con 0x7f496406ed20 2023-10-06T01:41:04.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640aea20 con 0x7f49640b23a0 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54962/0 (socket says 172.21.15.98:54962) 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 -- 172.21.15.98:0/4289058835 learned_addr learned my addr 172.21.15.98:0/4289058835 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f4963fff700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f496406ed20 0x7f496415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f4968f5f700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640b23a0 0x7f49640ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 -- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640b23a0 msgr2=0x7f49640ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640b23a0 0x7f49640ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 -- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f496406ed20 msgr2=0x7f496415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f496406ed20 0x7f496415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 -- 172.21.15.98:0/4289058835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c009a20 con 0x7f49640aee20 2023-10-06T01:41:04.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f49637fe700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4958006fd0 tx=0x7f4958017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495801f070 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4958002f70 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49580094f0 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800b820 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.529+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496416d7c0 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.530+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496416d950 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.531+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4958023070 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.531+0000 7f494bfff700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 0x7f49500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.531+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6839+0+0 (secure 0 0 0) 0x7f4958085cc0 con 0x7f49640aee20 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.531+0000 7f4963fff700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 0x7f49500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:04.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.531+0000 7f4963fff700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 0x7f49500693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4954007180 tx=0x7f4954007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:04.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.535+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4958052b80 con 0x7f49640aee20 2023-10-06T01:41:04.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:04.759+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true} v 0) v1 -- 0x7f4964064ad0 con 0x7f49640aee20 2023-10-06T01:41:05.056 INFO:tasks.ceph.mon.a.smithi098.stderr:2023-10-06T01:41:05.054+0000 7f8a67e63700 -1 log_channel(cluster) log [ERR] : Health check update: 2 filesystems are offline (MDS_ALL_DOWN) 2023-10-06T01:41:05.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.059+0000 7f494bfff700 1 -- 172.21.15.98:0/4289058835 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true}]=0 new fs with metadata pool 8 and data pool 7 v21) v1 ==== 241+0+0 (secure 0 0 0) 0x7f4958015100 con 0x7f49640aee20 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.064+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 msgr2=0x7f49500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.064+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 0x7f49500693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4954007180 tx=0x7f4954007ef0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.064+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 msgr2=0x7f496416dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.064+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4958006fd0 tx=0x7f4958017000 comp rx=0 tx=0).stop 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 shutdown_connections 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4950066f30 0x7f49500693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49640b23a0 0x7f49640ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f496406ed20 0x7f496415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 --2- 172.21.15.98:0/4289058835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49640aee20 0x7f496416dc50 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4958006fd0 tx=0x7f4958017000 comp rx=0 tx=0).stop 2023-10-06T01:41:05.067 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.065+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 >> 172.21.15.98:0/4289058835 conn(0x7f4964068330 msgr2=0x7f4964156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.069+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 shutdown_connections 2023-10-06T01:41:05.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.069+0000 7f496a9c2700 1 -- 172.21.15.98:0/4289058835 wait complete. 2023-10-06T01:41:05.073 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 8 and data pool 7 2023-10-06T01:41:05.086 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:05.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.199+0000 7fb3e548e700 1 Processor -- start 2023-10-06T01:41:05.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.199+0000 7fb3e548e700 1 -- start start 2023-10-06T01:41:05.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3e548e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3e01595a0 0x7fb3e00aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3e548e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 0x7fb3e00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3e548e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00b0ca0 con 0x7fb3e00b49e0 2023-10-06T01:41:05.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3e548e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00b0f80 con 0x7fb3e01595a0 2023-10-06T01:41:05.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3e548e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00b1260 con 0x7fb3e00b07a0 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3e01595a0 0x7fb3e00aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3d67fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb3e00b49e0 0x7fb3e00aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55280/0 (socket says 172.21.15.98:55280) 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3d67fc700 1 -- 172.21.15.98:0/3515612534 learned_addr learned my addr 172.21.15.98:0/3515612534 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3df7fe700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 0x7fb3e00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 -- 172.21.15.98:0/3515612534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 msgr2=0x7fb3e00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 0x7fb3e00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 -- 172.21.15.98:0/3515612534 >> v1:172.21.15.98:6789/0 conn(0x7fb3e00b49e0 legacy=0x7fb3e00aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 -- 172.21.15.98:0/3515612534 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00b1540 con 0x7fb3e01595a0 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3de7fc700 1 -- 172.21.15.98:0/3515612534 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660852807 0 0) 0x7fb3e00b0ca0 con 0x7fb3e00b49e0 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3df7fe700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 0x7fb3e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:05.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.200+0000 7fb3deffd700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e01595a0 0x7fb3e00aa570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009900 tx=0x7fb3d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f765469c56758a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3de7fc700 1 -- 172.21.15.98:0/3515612534 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0009ea0 con 0x7fb3e01595a0 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3de7fc700 1 -- 172.21.15.98:0/3515612534 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0005c10 con 0x7fb3e01595a0 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3de7fc700 1 -- 172.21.15.98:0/3515612534 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000b610 con 0x7fb3e01595a0 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 -- 172.21.15.98:0/3515612534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e01595a0 msgr2=0x7fb3e00aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e01595a0 0x7fb3e00aa570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009900 tx=0x7fb3d0009120 comp rx=0 tx=0).stop 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 -- 172.21.15.98:0/3515612534 shutdown_connections 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e00b07a0 0x7fb3e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 --2- 172.21.15.98:0/3515612534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e01595a0 0x7fb3e00aa570 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 -- 172.21.15.98:0/3515612534 >> 172.21.15.98:0/3515612534 conn(0x7fb3e0099830 msgr2=0x7fb3e006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 -- 172.21.15.98:0/3515612534 shutdown_connections 2023-10-06T01:41:05.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.201+0000 7fb3e548e700 1 -- 172.21.15.98:0/3515612534 wait complete. 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.202+0000 7fb3e548e700 1 Processor -- start 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.202+0000 7fb3e548e700 1 -- start start 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e00b49e0 0x7fb3e009da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 0x7fb3e0168f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e009dfb0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e009a120 con 0x7fb3e00b49e0 2023-10-06T01:41:05.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3e548e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e015aa60 con 0x7fb3e01595a0 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54984/0 (socket says 172.21.15.98:54984) 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 -- 172.21.15.98:0/2575157683 learned_addr learned my addr 172.21.15.98:0/2575157683 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3df7fe700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 0x7fb3e0168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3d67fc700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e00b49e0 0x7fb3e009da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 -- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 msgr2=0x7fb3e0168f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 0x7fb3e0168f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 -- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e00b49e0 msgr2=0x7fb3e009da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e00b49e0 0x7fb3e009da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3deffd700 1 -- 172.21.15.98:0/2575157683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00093c0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.203+0000 7fb3df7fe700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 0x7fb3e0168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:05.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3deffd700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0067c60 tx=0x7fb3d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0006590 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d00066f0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e015ad40 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001b480 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.204+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e015b300 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.205+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e0064ad0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.205+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3d001b7f0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.205+0000 7fb3d77fe700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 0x7fb3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.206+0000 7fb3d67fc700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 0x7fb3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.206+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fb3d0086c10 con 0x7fb3e00b07a0 2023-10-06T01:41:05.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.206+0000 7fb3d67fc700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 0x7fb3c00693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00b4550 tx=0x7fb3c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.210+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d00539c0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.395 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.393+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e00b1820 con 0x7fb3e00b07a0 2023-10-06T01:41:05.396 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.394+0000 7fb3d77fe700 1 -- 172.21.15.98:0/2575157683 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18940 (secure 0 0 0) 0x7fb3d0056fe0 con 0x7fb3e00b07a0 2023-10-06T01:41:05.396 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:05.397 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":43,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:04.761476+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:05.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.396+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 msgr2=0x7fb3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.396+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 0x7fb3c00693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00b4550 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.396+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 msgr2=0x7fb3e015a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.396+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0067c60 tx=0x7fb3d0016000 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 shutdown_connections 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb3c0066f30 0x7fb3c00693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e01595a0 0x7fb3e0168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3e00b07a0 0x7fb3e015a380 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 --2- 172.21.15.98:0/2575157683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e00b49e0 0x7fb3e009da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 >> 172.21.15.98:0/2575157683 conn(0x7fb3e0099830 msgr2=0x7fb3e0068400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 shutdown_connections 2023-10-06T01:41:05.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.397+0000 7fb3e548e700 1 -- 172.21.15.98:0/2575157683 wait complete. 2023-10-06T01:41:05.411 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:05.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.524+0000 7ff18e0b4700 1 Processor -- start 2023-10-06T01:41:05.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.524+0000 7ff18e0b4700 1 -- start start 2023-10-06T01:41:05.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff18e0b4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1880b49e0 0x7ff1880aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff18e0b4700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff1880b07a0 0x7ff1880b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff18e0b4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1880b0ca0 con 0x7ff1881595a0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff18e0b4700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880b0f80 con 0x7ff1880b49e0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff18e0b4700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880b1260 con 0x7ff1880b07a0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff1877fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff1881595a0 0x7ff1880aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55288/0 (socket says 172.21.15.98:55288) 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff1877fe700 1 -- 172.21.15.98:0/1904471762 learned_addr learned my addr 172.21.15.98:0/1904471762 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.525+0000 7ff186ffd700 1 --2- 172.21.15.98:0/1904471762 >> v2:172.21.15.98:3300/0 conn(0x7ff1880b49e0 0x7ff1880aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff1867fc700 1 -- 172.21.15.98:0/1904471762 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645329925 0 0) 0x7ff1880b0ca0 con 0x7ff1881595a0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff1867fc700 1 -- 172.21.15.98:0/1904471762 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003570 con 0x7ff1881595a0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff187fff700 1 --2- 172.21.15.98:0/1904471762 >> v2:172.21.15.184:3300/0 conn(0x7ff1880b07a0 0x7ff1880b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff186ffd700 1 -- 172.21.15.98:0/1904471762 >> v2:172.21.15.184:3300/0 conn(0x7ff1880b07a0 msgr2=0x7ff1880b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:05.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff186ffd700 1 --2- 172.21.15.98:0/1904471762 >> v2:172.21.15.184:3300/0 conn(0x7ff1880b07a0 0x7ff1880b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff186ffd700 1 -- 172.21.15.98:0/1904471762 >> v1:172.21.15.98:6789/0 conn(0x7ff1881595a0 legacy=0x7ff1880aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff186ffd700 1 -- 172.21.15.98:0/1904471762 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880b1540 con 0x7ff1880b49e0 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff186ffd700 1 --2- 172.21.15.98:0/1904471762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff1880aac80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0098e0 tx=0x7ff17c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fccde50470ff88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff1867fc700 1 -- 172.21.15.98:0/1904471762 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c011630 con 0x7ff1880b49e0 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff1867fc700 1 -- 172.21.15.98:0/1904471762 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c011790 con 0x7ff1880b49e0 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff1867fc700 1 -- 172.21.15.98:0/1904471762 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c011b10 con 0x7ff1880b49e0 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 -- 172.21.15.98:0/1904471762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 msgr2=0x7ff1880aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/1904471762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff1880aac80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0098e0 tx=0x7ff17c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 -- 172.21.15.98:0/1904471762 shutdown_connections 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/1904471762 >> v2:172.21.15.184:3300/0 conn(0x7ff1880b07a0 0x7ff1880b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.530 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/1904471762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff1880aac80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.526+0000 7ff18e0b4700 1 -- 172.21.15.98:0/1904471762 >> 172.21.15.98:0/1904471762 conn(0x7ff188099830 msgr2=0x7ff18806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.527+0000 7ff18e0b4700 1 -- 172.21.15.98:0/1904471762 shutdown_connections 2023-10-06T01:41:05.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.527+0000 7ff18e0b4700 1 -- 172.21.15.98:0/1904471762 wait complete. 2023-10-06T01:41:05.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 Processor -- start 2023-10-06T01:41:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 -- start start 2023-10-06T01:41:05.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 0x7ff18815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1881595a0 0x7ff18809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18809acd0 con 0x7ff1880b49e0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18809a130 con 0x7ff1880b07a0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.528+0000 7ff18e0b4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18809a3a0 con 0x7ff1881595a0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55002/0 (socket says 172.21.15.98:55002) 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 -- 172.21.15.98:0/3921610075 learned_addr learned my addr 172.21.15.98:0/3921610075 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff1877fe700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 0x7ff18815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff187fff700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1881595a0 0x7ff18809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 -- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1881595a0 msgr2=0x7ff18809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1881595a0 0x7ff18809bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 -- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 msgr2=0x7ff18815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 0x7ff18815afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 -- 172.21.15.98:0/3921610075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c009420 con 0x7ff1880b49e0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff1877fe700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 0x7ff18815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff186ffd700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff188067c60 tx=0x7ff17c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c011b10 con 0x7ff1880b49e0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c000c90 con 0x7ff1880b49e0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18809a680 con 0x7ff1880b49e0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c01a680 con 0x7ff1880b49e0 2023-10-06T01:41:05.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.529+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18815b4e0 con 0x7ff1880b49e0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.530+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18809ae40 con 0x7ff1880b49e0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.530+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff17c01aab0 con 0x7ff1880b49e0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.531+0000 7ff16f7fe700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.531+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6917+0+0 (secure 0 0 0) 0x7ff17c086780 con 0x7ff1880b49e0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.531+0000 7ff1877fe700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.531+0000 7ff1877fe700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.535+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff17c053530 con 0x7ff1880b49e0 2023-10-06T01:41:05.720 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.719+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff188064ad0 con 0x7ff1880b49e0 2023-10-06T01:41:05.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.720+0000 7ff16f7fe700 1 -- 172.21.15.98:0/3921610075 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18940 (secure 0 0 0) 0x7ff17c056b50 con 0x7ff1880b49e0 2023-10-06T01:41:05.722 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:05.722 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":43,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:04.761476+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 msgr2=0x7ff1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).stop 2023-10-06T01:41:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 msgr2=0x7ff18809b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff188067c60 tx=0x7ff17c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 shutdown_connections 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1881595a0 0x7ff18809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1880b07a0 0x7ff18815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 --2- 172.21.15.98:0/3921610075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880b49e0 0x7ff18809b690 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 >> 172.21.15.98:0/3921610075 conn(0x7ff188099830 msgr2=0x7ff18815d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 shutdown_connections 2023-10-06T01:41:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.722+0000 7ff18e0b4700 1 -- 172.21.15.98:0/3921610075 wait complete. 2023-10-06T01:41:05.736 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs standby_count_wanted 0 2023-10-06T01:41:05.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.851+0000 7f26a3f4e700 1 Processor -- start 2023-10-06T01:41:05.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a3f4e700 1 -- start start 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a3f4e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f269c159510 0x7f269c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a3f4e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c0cbf20 con 0x7f269c1598e0 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a3f4e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c0cf610 con 0x7f269c0cbae0 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a3f4e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c0cf8f0 con 0x7f269c159510 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a14e9700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f269c1598e0 0x7f269c0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55292/0 (socket says 172.21.15.98:55292) 2023-10-06T01:41:05.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.852+0000 7f26a14e9700 1 -- 172.21.15.98:0/807628338 learned_addr learned my addr 172.21.15.98:0/807628338 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a1cea700 1 --2- 172.21.15.98:0/807628338 >> v2:172.21.15.184:3300/0 conn(0x7f269c159510 0x7f269c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670406266 0 0) 0x7f269c0cbf20 con 0x7f269c1598e0 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2688003570 con 0x7f269c1598e0 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589389458 0 0) 0x7f2688003570 con 0x7f269c1598e0 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 >> v2:172.21.15.184:3300/0 conn(0x7f269c159510 msgr2=0x7f269c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 --2- 172.21.15.98:0/807628338 >> v2:172.21.15.184:3300/0 conn(0x7f269c159510 0x7f269c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 >> v1:172.21.15.184:6789/0 conn(0x7f269c0cbae0 legacy=0x7f269c0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c0cfbd0 con 0x7f269c1598e0 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f268c001470 con 0x7f269c1598e0 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 >> v1:172.21.15.98:6789/0 conn(0x7f269c1598e0 legacy=0x7f269c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2688003a60 0x7f2688003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2688004810 0x7f2688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 0x7f268800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0cf8f0 con 0x7f26880052e0 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268800c190 con 0x7f2688004810 2023-10-06T01:41:05.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268800c350 con 0x7f2688003a60 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a1cea700 1 --2- 172.21.15.98:0/807628338 >> v2:172.21.15.184:3300/0 conn(0x7f269c159510 0x7f269c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 0x7f268800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a14e9700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2688004810 0x7f2688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a1cea700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2688003a60 0x7f2688003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 -- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2688003a60 msgr2=0x7f2688003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2688003a60 0x7f2688003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 -- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2688004810 msgr2=0x7f2688004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:05.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2688004810 0x7f2688004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.853+0000 7f26a24eb700 1 -- 172.21.15.98:0/807628338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688003570 con 0x7f26880052e0 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a24eb700 1 -- 172.21.15.98:0/807628338 reap_dead start 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a24eb700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 0x7f268800ba60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26940018b0 tx=0x7f2694007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5a622b905eaf99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694012420 con 0x7f26880052e0 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940125a0 con 0x7f26880052e0 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a0ce8700 1 -- 172.21.15.98:0/807628338 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694012960 con 0x7f26880052e0 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 -- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 msgr2=0x7f268800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 0x7f268800ba60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26940018b0 tx=0x7f2694007d30 comp rx=0 tx=0).stop 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 -- 172.21.15.98:0/807628338 shutdown_connections 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/807628338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26880052e0 0x7f268800ba60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 -- 172.21.15.98:0/807628338 >> 172.21.15.98:0/807628338 conn(0x7f269c089e90 msgr2=0x7f269c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:05.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 -- 172.21.15.98:0/807628338 shutdown_connections 2023-10-06T01:41:05.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.854+0000 7f26a3f4e700 1 -- 172.21.15.98:0/807628338 wait complete. 2023-10-06T01:41:05.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.855+0000 7f26a3f4e700 1 Processor -- start 2023-10-06T01:41:05.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 -- start start 2023-10-06T01:41:05.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880052e0 0x7f269c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c15b1f0 0x7f269c149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c15b650 con 0x7f269c0b3cd0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0c7be0 con 0x7f269c15b1f0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a3f4e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0c7ec0 con 0x7f26880052e0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a14e9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a14e9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55030/0 (socket says 172.21.15.98:55030) 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a14e9700 1 -- 172.21.15.98:0/31562140 learned_addr learned my addr 172.21.15.98:0/31562140 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.856+0000 7f26a1cea700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880052e0 0x7f269c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a24eb700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c15b1f0 0x7f269c149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 -- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880052e0 msgr2=0x7f269c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880052e0 0x7f269c0c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 -- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c15b1f0 msgr2=0x7f269c149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c15b1f0 0x7f269c149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 -- 172.21.15.98:0/31562140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c0c81a0 con 0x7f269c0b3cd0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a14e9700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f268c007390 tx=0x7f268c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c018030 con 0x7f269c0b3cd0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00f870 con 0x7f269c0b3cd0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c0c8480 con 0x7f269c0b3cd0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00fc50 con 0x7f269c0b3cd0 2023-10-06T01:41:05.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.857+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c0b4140 con 0x7f269c0b3cd0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.858+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c064ad0 con 0x7f269c0b3cd0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.858+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f268c01b070 con 0x7f269c0b3cd0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.859+0000 7f2691ffb700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 0x7f26800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.859+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f268c019070 con 0x7f269c0b3cd0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.859+0000 7f26a1cea700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 0x7f26800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.859+0000 7f26a1cea700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 0x7f26800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26980095e0 tx=0x7f269800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:05.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:05.863+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f268c0516e0 con 0x7f269c0b3cd0 2023-10-06T01:41:06.090 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:06.089+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f269c0cfeb0 con 0x7f269c0b3cd0 2023-10-06T01:41:07.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.073+0000 7f2691ffb700 1 -- 172.21.15.98:0/31562140 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f268c054d00 con 0x7f269c0b3cd0 2023-10-06T01:41:07.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 msgr2=0x7f26800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 0x7f26800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26980095e0 tx=0x7f269800b040 comp rx=0 tx=0).stop 2023-10-06T01:41:07.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 msgr2=0x7f269c15acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f268c007390 tx=0x7f268c001e50 comp rx=0 tx=0).stop 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 shutdown_connections 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2680066f30 0x7f26800693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c15b1f0 0x7f269c149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880052e0 0x7f269c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 --2- 172.21.15.98:0/31562140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269c0b3cd0 0x7f269c15acb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 >> 172.21.15.98:0/31562140 conn(0x7f269c089e90 msgr2=0x7f269c0687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.076+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 shutdown_connections 2023-10-06T01:41:07.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.077+0000 7f26a3f4e700 1 -- 172.21.15.98:0/31562140 wait complete. 2023-10-06T01:41:07.093 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs allow_standby_replay false 2023-10-06T01:41:07.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.214+0000 7f37e9693700 1 Processor -- start 2023-10-06T01:41:07.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.214+0000 7f37e9693700 1 -- start start 2023-10-06T01:41:07.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e9693700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37e40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e9693700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37e40cbae0 0x7f37e40cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e9693700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40cbf00 con 0x7f37e41598e0 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e9693700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40cf610 con 0x7f37e40cbae0 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e9693700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cf8f0 con 0x7f37e4159510 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37da7fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f37e41598e0 0x7f37e40ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55302/0 (socket says 172.21.15.98:55302) 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37da7fc700 1 -- 172.21.15.98:0/707754050 learned_addr learned my addr 172.21.15.98:0/707754050 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e37fe700 1 --2- 172.21.15.98:0/707754050 >> v2:172.21.15.184:3300/0 conn(0x7f37e40cbae0 0x7f37e40cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630142147 0 0) 0x7f37e40cbf00 con 0x7f37e41598e0 2023-10-06T01:41:07.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37e40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d0003570 con 0x7f37e41598e0 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.215+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077430896 0 0) 0x7f37d0003570 con 0x7f37e41598e0 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 msgr2=0x7f37e40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37e40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 >> v2:172.21.15.184:3300/0 conn(0x7f37e40cbae0 msgr2=0x7f37e40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 --2- 172.21.15.98:0/707754050 >> v2:172.21.15.184:3300/0 conn(0x7f37e40cbae0 0x7f37e40cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40cfbd0 con 0x7f37e41598e0 2023-10-06T01:41:07.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f37cc001470 con 0x7f37e41598e0 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 >> v1:172.21.15.98:6789/0 conn(0x7f37e41598e0 legacy=0x7f37e40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37d0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d0004d50 0x7f37d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cf610 con 0x7f37d0003fe0 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cf8f0 con 0x7f37d0004d50 2023-10-06T01:41:07.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d000ba90 con 0x7f37e4159510 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e37fe700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d0004d50 0x7f37d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37d0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 -- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 msgr2=0x7f37d0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37d0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 -- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d0004d50 msgr2=0x7f37d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:07.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d0004d50 0x7f37d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 -- 172.21.15.98:0/707754050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0003570 con 0x7f37d0003fe0 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e37fe700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d0004d50 0x7f37d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e4159510 0x7f37d0003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37da7fc700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37d0004810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c390 tx=0x7f37cc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6615cf342b5731fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00cc00 con 0x7f37d0003fe0 2023-10-06T01:41:07.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc00cd80 con 0x7f37d0003fe0 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.216+0000 7f37e27fc700 1 -- 172.21.15.98:0/707754050 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc005b70 con 0x7f37d0003fe0 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 -- 172.21.15.98:0/707754050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 msgr2=0x7f37d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 --2- 172.21.15.98:0/707754050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37d0004810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c390 tx=0x7f37cc001e90 comp rx=0 tx=0).stop 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e37fe700 1 -- 172.21.15.98:0/707754050 reap_dead start 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 -- 172.21.15.98:0/707754050 shutdown_connections 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 -- 172.21.15.98:0/707754050 >> 172.21.15.98:0/707754050 conn(0x7f37e4089e90 msgr2=0x7f37e40997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 -- 172.21.15.98:0/707754050 shutdown_connections 2023-10-06T01:41:07.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.217+0000 7f37e9693700 1 -- 172.21.15.98:0/707754050 wait complete. 2023-10-06T01:41:07.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.218+0000 7f37e9693700 1 Processor -- start 2023-10-06T01:41:07.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.218+0000 7f37e9693700 1 -- start start 2023-10-06T01:41:07.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 0x7f37e40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e40b6120 0x7f37e415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416e2a0 con 0x7f37d0003fe0 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416d690 con 0x7f37e40b4ec0 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416d970 con 0x7f37e40b6120 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55060/0 (socket says 172.21.15.98:55060) 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 -- 172.21.15.98:0/2838840463 learned_addr learned my addr 172.21.15.98:0/2838840463 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37da7fc700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 0x7f37e40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e37fe700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e40b6120 0x7f37e415cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 -- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e40b6120 msgr2=0x7f37e415cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e40b6120 0x7f37e415cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 -- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 msgr2=0x7f37e40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 0x7f37e40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 -- 172.21.15.98:0/2838840463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e416dc50 con 0x7f37d0003fe0 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37da7fc700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 0x7f37e40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e2ffd700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f37d4011860 tx=0x7f37d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400f700 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d400f860 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e416df30 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.219+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400fc00 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.220+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e4157be0 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.220+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e4064ad0 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.221+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37d4015460 con 0x7f37d0003fe0 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.221+0000 7f37db7fe700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 0x7f37c4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:07.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.221+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f37d40851f0 con 0x7f37d0003fe0 2023-10-06T01:41:07.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.221+0000 7f37da7fc700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 0x7f37c4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:07.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.222+0000 7f37da7fc700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 0x7f37c4069460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37cc001ed0 tx=0x7f37cc015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:07.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.225+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37d4051f00 con 0x7f37d0003fe0 2023-10-06T01:41:07.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:07.453+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f37e40cfeb0 con 0x7f37d0003fe0 2023-10-06T01:41:08.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.083+0000 7f37db7fe700 1 -- 172.21.15.98:0/2838840463 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 127+0+0 (secure 0 0 0) 0x7f37d4055520 con 0x7f37d0003fe0 2023-10-06T01:41:08.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 msgr2=0x7f37c4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 0x7f37c4069460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37cc001ed0 tx=0x7f37cc015000 comp rx=0 tx=0).stop 2023-10-06T01:41:08.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 msgr2=0x7f37e40b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f37d4011860 tx=0x7f37d4008040 comp rx=0 tx=0).stop 2023-10-06T01:41:08.087 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 shutdown_connections 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37c4066fb0 0x7f37c4069460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e40b6120 0x7f37e415cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37d0003fe0 0x7f37e40b4010 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 --2- 172.21.15.98:0/2838840463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e40b4ec0 0x7f37e40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 >> 172.21.15.98:0/2838840463 conn(0x7f37e4089e90 msgr2=0x7f37e4154600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 shutdown_connections 2023-10-06T01:41:08.088 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.085+0000 7f37e9693700 1 -- 172.21.15.98:0/2838840463 wait complete. 2023-10-06T01:41:08.100 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:08.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T01:41:08.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.214+0000 7fd27f6fd700 1 Processor -- start 2023-10-06T01:41:08.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.214+0000 7fd27f6fd700 1 -- start start 2023-10-06T01:41:08.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27f6fd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2780b49e0 0x7fd2780aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27f6fd700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2780b0ca0 con 0x7fd2781595a0 2023-10-06T01:41:08.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27f6fd700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2780b0f80 con 0x7fd2780b07a0 2023-10-06T01:41:08.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27f6fd700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780b1260 con 0x7fd2780b49e0 2023-10-06T01:41:08.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27cc98700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2780b49e0 0x7fd2780aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27d499700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd2781595a0 0x7fd2780aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55310/0 (socket says 172.21.15.98:55310) 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27d499700 1 -- 172.21.15.98:0/3754993761 learned_addr learned my addr 172.21.15.98:0/3754993761 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27cc98700 1 -- 172.21.15.98:0/3754993761 >> v1:172.21.15.184:6789/0 conn(0x7fd2780b07a0 legacy=0x7fd2780b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27cc98700 1 -- 172.21.15.98:0/3754993761 >> v1:172.21.15.98:6789/0 conn(0x7fd2781595a0 legacy=0x7fd2780aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27cc98700 1 -- 172.21.15.98:0/3754993761 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780b1540 con 0x7fd2780b49e0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd26ffff700 1 -- 172.21.15.98:0/3754993761 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734021323 0 0) 0x7fd2780b0ca0 con 0x7fd2781595a0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd27cc98700 1 --2- 172.21.15.98:0/3754993761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd2780aac80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd268009910 tx=0x7fd2680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1284a75f3c2fd15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd26ffff700 1 -- 172.21.15.98:0/3754993761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800b7e0 con 0x7fd2780b49e0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd26ffff700 1 -- 172.21.15.98:0/3754993761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26800b940 con 0x7fd2780b49e0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.215+0000 7fd26ffff700 1 -- 172.21.15.98:0/3754993761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800bce0 con 0x7fd2780b49e0 2023-10-06T01:41:08.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 -- 172.21.15.98:0/3754993761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 msgr2=0x7fd2780aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/3754993761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd2780aac80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd268009910 tx=0x7fd2680091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 -- 172.21.15.98:0/3754993761 shutdown_connections 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/3754993761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd2780aac80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 -- 172.21.15.98:0/3754993761 >> 172.21.15.98:0/3754993761 conn(0x7fd278099830 msgr2=0x7fd27806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 -- 172.21.15.98:0/3754993761 shutdown_connections 2023-10-06T01:41:08.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.216+0000 7fd27f6fd700 1 -- 172.21.15.98:0/3754993761 wait complete. 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.217+0000 7fd27f6fd700 1 Processor -- start 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.217+0000 7fd27f6fd700 1 -- start start 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2780b07a0 0x7fd27809a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 0x7fd27809d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27815be50 con 0x7fd2780b49e0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278156f80 con 0x7fd2781595a0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27f6fd700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27809b010 con 0x7fd2780b07a0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55082/0 (socket says 172.21.15.98:55082) 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 -- 172.21.15.98:0/2763091544 learned_addr learned my addr 172.21.15.98:0/2763091544 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27d499700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2780b07a0 0x7fd27809a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 -- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2780b07a0 msgr2=0x7fd27809a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27dc9a700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 0x7fd27809d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2780b07a0 0x7fd27809a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 -- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 msgr2=0x7fd27809d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 0x7fd27809d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27cc98700 1 -- 172.21.15.98:0/2763091544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009420 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.218+0000 7fd27dc9a700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 0x7fd27809d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd27cc98700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd2680093f0 tx=0x7fd268016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26801b680 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26801b7e0 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27809a770 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26801bba0 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.219+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27809ad30 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.220+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27815bfc0 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.220+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd26801c450 con 0x7fd2780b49e0 2023-10-06T01:41:08.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.220+0000 7fd26d7fa700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 0x7fd25c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.221+0000 7fd27d499700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 0x7fd25c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.221+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fd268086380 con 0x7fd2780b49e0 2023-10-06T01:41:08.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.221+0000 7fd27d499700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 0x7fd25c0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd274001730 tx=0x7fd274009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.225+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd27815bfc0 con 0x7fd2780b49e0 2023-10-06T01:41:08.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.449+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd27815bf90 con 0x7fd2780b49e0 2023-10-06T01:41:08.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.449+0000 7fd26d7fa700 1 -- 172.21.15.98:0/2763091544 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5660 (secure 0 0 0) 0x7fd268053130 con 0x7fd2780b49e0 2023-10-06T01:41:08.451 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:08.451 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:08.079169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.451+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 msgr2=0x7fd25c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.451+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 0x7fd25c0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd274001730 tx=0x7fd274009040 comp rx=0 tx=0).stop 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 msgr2=0x7fd27809b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd2680093f0 tx=0x7fd268016000 comp rx=0 tx=0).stop 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 shutdown_connections 2023-10-06T01:41:08.454 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd25c066f30 0x7fd25c0693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2781595a0 0x7fd27809d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2780b07a0 0x7fd27809a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 --2- 172.21.15.98:0/2763091544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2780b49e0 0x7fd27809b9a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 >> 172.21.15.98:0/2763091544 conn(0x7fd278099830 msgr2=0x7fd2780695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.455 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 shutdown_connections 2023-10-06T01:41:08.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.452+0000 7fd27f6fd700 1 -- 172.21.15.98:0/2763091544 wait complete. 2023-10-06T01:41:08.456 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2023-10-06T01:41:08.465 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:08.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.578+0000 7f4b8ffff700 1 Processor -- start 2023-10-06T01:41:08.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.578+0000 7f4b8ffff700 1 -- start start 2023-10-06T01:41:08.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8ffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b90159480 0x7f4b900b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b900b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900cb7e0 con 0x7f4b900b3610 2023-10-06T01:41:08.582 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8ffff700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b900b08a0 con 0x7f4b90159480 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900cb190 con 0x7f4b900b2140 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b90159480 0x7f4b900b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b90159480 0x7f4b900b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55092/0 (socket says 172.21.15.98:55092) 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 -- 172.21.15.98:0/1354124966 learned_addr learned my addr 172.21.15.98:0/1354124966 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8f7fe700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b900b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 -- 172.21.15.98:0/1354124966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 msgr2=0x7f4b900b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b900b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 -- 172.21.15.98:0/1354124966 >> v1:172.21.15.184:6789/0 conn(0x7f4b900b3610 legacy=0x7f4b900cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8effd700 1 -- 172.21.15.98:0/1354124966 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900cb470 con 0x7f4b90159480 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.579+0000 7f4b8f7fe700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b900b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:08.583 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8effd700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b90159480 0x7f4b900b30d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b800099b0 tx=0x7f4b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29805775abdb1655 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8e7fc700 1 -- 172.21.15.98:0/1354124966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000b820 con 0x7f4b90159480 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8e7fc700 1 -- 172.21.15.98:0/1354124966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8000b980 con 0x7f4b90159480 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8e7fc700 1 -- 172.21.15.98:0/1354124966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000bd20 con 0x7f4b90159480 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 -- 172.21.15.98:0/1354124966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b90159480 msgr2=0x7f4b900b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b90159480 0x7f4b900b30d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b800099b0 tx=0x7f4b80009250 comp rx=0 tx=0).stop 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 -- 172.21.15.98:0/1354124966 shutdown_connections 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b900b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/1354124966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b90159480 0x7f4b900b30d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 -- 172.21.15.98:0/1354124966 >> 172.21.15.98:0/1354124966 conn(0x7f4b9006ed20 msgr2=0x7f4b9006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 -- 172.21.15.98:0/1354124966 shutdown_connections 2023-10-06T01:41:08.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.580+0000 7f4b8ffff700 1 -- 172.21.15.98:0/1354124966 wait complete. 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.581+0000 7f4b8ffff700 1 Processor -- start 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 -- start start 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b90154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 0x7f4b900c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015d4a0 con 0x7f4b900b3610 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015d7b0 con 0x7f4b90159480 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b8ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015da90 con 0x7f4b900b2140 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b867fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b867fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55104/0 (socket says 172.21.15.98:55104) 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.582+0000 7f4b867fc700 1 -- 172.21.15.98:0/3111149334 learned_addr learned my addr 172.21.15.98:0/3111149334 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8effd700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b90154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8f7fe700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 0x7f4b900c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 -- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 msgr2=0x7f4b90154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b90154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 -- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 msgr2=0x7f4b900c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.589 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 0x7f4b900c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 -- 172.21.15.98:0/3111149334 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800094f0 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8effd700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b90154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8f7fe700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 0x7f4b900c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b867fc700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c5d0 tx=0x7f4b78007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78011590 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78011710 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9015dd70 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78011ad0 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.583+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9015e100 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.584+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9015e570 con 0x7f4b900b3610 2023-10-06T01:41:08.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.584+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b7801c460 con 0x7f4b900b3610 2023-10-06T01:41:08.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.585+0000 7f4b877fe700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.585+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f4b780847f0 con 0x7f4b900b3610 2023-10-06T01:41:08.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.585+0000 7f4b8effd700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.585+0000 7f4b8effd700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 0x7f4b700693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b800099b0 tx=0x7f4b80016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.589+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b9015e570 con 0x7f4b900b3610 2023-10-06T01:41:08.773 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.771+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b9015e570 con 0x7f4b900b3610 2023-10-06T01:41:08.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.772+0000 7f4b877fe700 1 -- 172.21.15.98:0/3111149334 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f4b9015e570 con 0x7f4b900b3610 2023-10-06T01:41:08.774 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:08.774 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":44,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:06.067115+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:08.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 msgr2=0x7f4b700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 0x7f4b700693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b800099b0 tx=0x7f4b80016000 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 msgr2=0x7f4b900c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c5d0 tx=0x7f4b78007f60 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 shutdown_connections 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b90159480 0x7f4b900c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b900b2140 0x7f4b90154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 --2- 172.21.15.98:0/3111149334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b900b3610 0x7f4b900c7310 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 >> 172.21.15.98:0/3111149334 conn(0x7f4b9006ed20 msgr2=0x7f4b90156820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 shutdown_connections 2023-10-06T01:41:08.777 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.774+0000 7f4b8ffff700 1 -- 172.21.15.98:0/3111149334 wait complete. 2023-10-06T01:41:08.788 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T01:41:08.788 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:08.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.901+0000 7f472d965700 1 Processor -- start 2023-10-06T01:41:08.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.901+0000 7f472d965700 1 -- start start 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d965700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47280b2850 0x7f47280cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d965700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280cb7e0 con 0x7f47280cbc70 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d965700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280b08a0 con 0x7f4728159480 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d965700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47280cb190 con 0x7f47280b2850 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d164700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47280b2850 0x7f47280cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f471ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f47280cbc70 0x7f47280b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55324/0 (socket says 172.21.15.98:55324) 2023-10-06T01:41:08.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f471ffff700 1 -- 172.21.15.98:0/1839770363 learned_addr learned my addr 172.21.15.98:0/1839770363 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d164700 1 -- 172.21.15.98:0/1839770363 >> v1:172.21.15.184:6789/0 conn(0x7f4728159480 legacy=0x7f47280b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d164700 1 -- 172.21.15.98:0/1839770363 >> v1:172.21.15.98:6789/0 conn(0x7f47280cbc70 legacy=0x7f47280b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f472d164700 1 -- 172.21.15.98:0/1839770363 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280cb470 con 0x7f47280b2850 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.902+0000 7f471f7fe700 1 -- 172.21.15.98:0/1839770363 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924327590 0 0) 0x7f47280cb7e0 con 0x7f47280cbc70 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d164700 1 --2- 172.21.15.98:0/1839770363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280b2850 0x7f47280cc0c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47200098e0 tx=0x7f47200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e303d33b1435fb0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f471f7fe700 1 -- 172.21.15.98:0/1839770363 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720009e80 con 0x7f47280b2850 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f471f7fe700 1 -- 172.21.15.98:0/1839770363 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472000b860 con 0x7f47280b2850 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f471f7fe700 1 -- 172.21.15.98:0/1839770363 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000bbe0 con 0x7f47280b2850 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 -- 172.21.15.98:0/1839770363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280b2850 msgr2=0x7f47280cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 --2- 172.21.15.98:0/1839770363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280b2850 0x7f47280cc0c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47200098e0 tx=0x7f47200091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 -- 172.21.15.98:0/1839770363 shutdown_connections 2023-10-06T01:41:08.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 --2- 172.21.15.98:0/1839770363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280b2850 0x7f47280cc0c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 -- 172.21.15.98:0/1839770363 >> 172.21.15.98:0/1839770363 conn(0x7f472806ed20 msgr2=0x7f472806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:08.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 -- 172.21.15.98:0/1839770363 shutdown_connections 2023-10-06T01:41:08.907 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.903+0000 7f472d965700 1 -- 172.21.15.98:0/1839770363 wait complete. 2023-10-06T01:41:08.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.904+0000 7f472d965700 1 Processor -- start 2023-10-06T01:41:08.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.904+0000 7f472d965700 1 -- start start 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47280b2850 0x7f472816ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 0x7f47280c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280c7ce0 con 0x7f47280cbc70 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728169c60 con 0x7f4728159480 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d965700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280c73b0 con 0x7f47280b2850 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f471ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f471ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55108/0 (socket says 172.21.15.98:55108) 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f471ffff700 1 -- 172.21.15.98:0/1310504018 learned_addr learned my addr 172.21.15.98:0/1310504018 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:08.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472c963700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47280b2850 0x7f472816ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.905+0000 7f472d164700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 0x7f47280c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 -- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47280b2850 msgr2=0x7f472816ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47280b2850 0x7f472816ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 -- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 msgr2=0x7f47280c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 0x7f47280c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 -- 172.21.15.98:0/1310504018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009420 con 0x7f47280cbc70 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f472d164700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 0x7f47280c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471ffff700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f471800c7e0 tx=0x7f4718007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718005b30 con 0x7f47280cbc70 2023-10-06T01:41:08.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718005c90 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280c7690 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718007300 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.906+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472816bfb0 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.907+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472805f270 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.907+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f471801b400 con 0x7f47280cbc70 2023-10-06T01:41:08.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.908+0000 7f471cff9700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:08.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.908+0000 7f472c963700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:08.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.908+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f47180844b0 con 0x7f47280cbc70 2023-10-06T01:41:08.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.908+0000 7f472c963700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 0x7f47140693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4724007560 tx=0x7f4724009760 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:08.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:08.912+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47180512e0 con 0x7f47280cbc70 2023-10-06T01:41:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.089+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4728159850 con 0x7f4714066f30 2023-10-06T01:41:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.099+0000 7f471cff9700 1 -- 172.21.15.98:0/1310504018 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+390293 (secure 0 0 0) 0x7f4728159850 con 0x7f4714066f30 2023-10-06T01:41:09.105 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 msgr2=0x7f47140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 0x7f47140693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4724007560 tx=0x7f4724009760 comp rx=0 tx=0).stop 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 msgr2=0x7f47280c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f471800c7e0 tx=0x7f4718007df0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 shutdown_connections 2023-10-06T01:41:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4728159480 0x7f47280c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47280b2850 0x7f472816ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 --2- 172.21.15.98:0/1310504018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47280cbc70 0x7f47280c5f60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 >> 172.21.15.98:0/1310504018 conn(0x7f472806ed20 msgr2=0x7f4728157060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 shutdown_connections 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.103+0000 7f472d965700 1 -- 172.21.15.98:0/1310504018 wait complete. 2023-10-06T01:41:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:09.116 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-10-06T01:41:07.802543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":250,"acting":250,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395640,"kb_used_data":17336,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579080,"statfs":{"total":773094113280,"available":770640977920,"internally_reserved":0,"allocated":17752064,"data_stored":15606319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-90,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.752233"},"pg_stats":[{"pgid":"7.3f","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164681+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:56.164681+0000","last_peered":"2023-10-06T01:40:56.164681+0000","last_clean":"2023-10-06T01:40:56.164681+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:56.164681+0000","last_undegraded":"2023-10-06T01:40:56.164681+0000","last_fullsized":"2023-10-06T01:40:56.164681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068055+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:41:05.068055+0000","last_peered":"2023-10-06T01:41:05.068055+0000","last_clean":"2023-10-06T01:41:05.068055+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:41:05.068055+0000","last_undegraded":"2023-10-06T01:41:05.068055+0000","last_fullsized":"2023-10-06T01:41:05.068055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062731+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:41:05.062731+0000","last_peered":"2023-10-06T01:41:05.062731+0000","last_clean":"2023-10-06T01:41:05.062731+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:41:05.062731+0000","last_undegraded":"2023-10-06T01:41:05.062731+0000","last_fullsized":"2023-10-06T01:41:05.062731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165280+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:56.165280+0000","last_peered":"2023-10-06T01:40:56.165280+0000","last_clean":"2023-10-06T01:40:56.165280+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:56.165280+0000","last_undegraded":"2023-10-06T01:40:56.165280+0000","last_fullsized":"2023-10-06T01:40:56.165280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067188+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:41:05.067188+0000","last_peered":"2023-10-06T01:41:05.067188+0000","last_clean":"2023-10-06T01:41:05.067188+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:41:05.067188+0000","last_undegraded":"2023-10-06T01:41:05.067188+0000","last_fullsized":"2023-10-06T01:41:05.067188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067168+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:41:05.067168+0000","last_peered":"2023-10-06T01:41:05.067168+0000","last_clean":"2023-10-06T01:41:05.067168+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:41:05.067168+0000","last_undegraded":"2023-10-06T01:41:05.067168+0000","last_fullsized":"2023-10-06T01:41:05.067168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"36'5","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164751+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:56.164751+0000","last_peered":"2023-10-06T01:40:56.164751+0000","last_clean":"2023-10-06T01:40:56.164751+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:56.164751+0000","last_undegraded":"2023-10-06T01:40:56.164751+0000","last_fullsized":"2023-10-06T01:40:56.164751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164949+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:56.164949+0000","last_peered":"2023-10-06T01:40:56.164949+0000","last_clean":"2023-10-06T01:40:56.164949+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:56.164949+0000","last_undegraded":"2023-10-06T01:40:56.164949+0000","last_fullsized":"2023-10-06T01:40:56.164949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"36'11","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165851+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:56.165851+0000","last_peered":"2023-10-06T01:40:56.165851+0000","last_clean":"2023-10-06T01:40:56.165851+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:56.165851+0000","last_undegraded":"2023-10-06T01:40:56.165851+0000","last_fullsized":"2023-10-06T01:40:56.165851+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"36'15","reported_seq":27,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165484+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:56.165484+0000","last_peered":"2023-10-06T01:40:56.165484+0000","last_clean":"2023-10-06T01:40:56.165484+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:56.165484+0000","last_undegraded":"2023-10-06T01:40:56.165484+0000","last_fullsized":"2023-10-06T01:40:56.165484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165051+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:56.165051+0000","last_peered":"2023-10-06T01:40:56.165051+0000","last_clean":"2023-10-06T01:40:56.165051+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:56.165051+0000","last_undegraded":"2023-10-06T01:40:56.165051+0000","last_fullsized":"2023-10-06T01:40:56.165051+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066867+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:41:05.066867+0000","last_peered":"2023-10-06T01:41:05.066867+0000","last_clean":"2023-10-06T01:41:05.066867+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:41:05.066867+0000","last_undegraded":"2023-10-06T01:41:05.066867+0000","last_fullsized":"2023-10-06T01:41:05.066867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"36'10","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059981+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:41:05.059981+0000","last_peered":"2023-10-06T01:41:05.059981+0000","last_clean":"2023-10-06T01:41:05.059981+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:41:05.059981+0000","last_undegraded":"2023-10-06T01:41:05.059981+0000","last_fullsized":"2023-10-06T01:41:05.059981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165162+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:56.165162+0000","last_peered":"2023-10-06T01:40:56.165162+0000","last_clean":"2023-10-06T01:40:56.165162+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:56.165162+0000","last_undegraded":"2023-10-06T01:40:56.165162+0000","last_fullsized":"2023-10-06T01:40:56.165162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066663+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:41:05.066663+0000","last_peered":"2023-10-06T01:41:05.066663+0000","last_clean":"2023-10-06T01:41:05.066663+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:41:05.066663+0000","last_undegraded":"2023-10-06T01:41:05.066663+0000","last_fullsized":"2023-10-06T01:41:05.066663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"36'9","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040773+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:56.040773+0000","last_peered":"2023-10-06T01:40:56.040773+0000","last_clean":"2023-10-06T01:40:56.040773+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:56.040773+0000","last_undegraded":"2023-10-06T01:40:56.040773+0000","last_fullsized":"2023-10-06T01:40:56.040773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063591+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:41:05.063591+0000","last_peered":"2023-10-06T01:41:05.063591+0000","last_clean":"2023-10-06T01:41:05.063591+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:41:05.063591+0000","last_undegraded":"2023-10-06T01:41:05.063591+0000","last_fullsized":"2023-10-06T01:41:05.063591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062553+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:41:05.062553+0000","last_peered":"2023-10-06T01:41:05.062553+0000","last_clean":"2023-10-06T01:41:05.062553+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:41:05.062553+0000","last_undegraded":"2023-10-06T01:41:05.062553+0000","last_fullsized":"2023-10-06T01:41:05.062553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041241+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:56.041241+0000","last_peered":"2023-10-06T01:40:56.041241+0000","last_clean":"2023-10-06T01:40:56.041241+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:56.041241+0000","last_undegraded":"2023-10-06T01:40:56.041241+0000","last_fullsized":"2023-10-06T01:40:56.041241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059961+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:41:05.059961+0000","last_peered":"2023-10-06T01:41:05.059961+0000","last_clean":"2023-10-06T01:41:05.059961+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:41:05.059961+0000","last_undegraded":"2023-10-06T01:41:05.059961+0000","last_fullsized":"2023-10-06T01:41:05.059961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067584+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:41:05.067584+0000","last_peered":"2023-10-06T01:41:05.067584+0000","last_clean":"2023-10-06T01:41:05.067584+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:41:05.067584+0000","last_undegraded":"2023-10-06T01:41:05.067584+0000","last_fullsized":"2023-10-06T01:41:05.067584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165192+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:56.165192+0000","last_peered":"2023-10-06T01:40:56.165192+0000","last_clean":"2023-10-06T01:40:56.165192+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:56.165192+0000","last_undegraded":"2023-10-06T01:40:56.165192+0000","last_fullsized":"2023-10-06T01:40:56.165192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"36'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040905+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:56.040905+0000","last_peered":"2023-10-06T01:40:56.040905+0000","last_clean":"2023-10-06T01:40:56.040905+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:56.040905+0000","last_undegraded":"2023-10-06T01:40:56.040905+0000","last_fullsized":"2023-10-06T01:40:56.040905+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040858+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:56.040858+0000","last_peered":"2023-10-06T01:40:56.040858+0000","last_clean":"2023-10-06T01:40:56.040858+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:56.040858+0000","last_undegraded":"2023-10-06T01:40:56.040858+0000","last_fullsized":"2023-10-06T01:40:56.040858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"36'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041744+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:56.041744+0000","last_peered":"2023-10-06T01:40:56.041744+0000","last_clean":"2023-10-06T01:40:56.041744+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:56.041744+0000","last_undegraded":"2023-10-06T01:40:56.041744+0000","last_fullsized":"2023-10-06T01:40:56.041744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"36'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059914+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:41:05.059914+0000","last_peered":"2023-10-06T01:41:05.059914+0000","last_clean":"2023-10-06T01:41:05.059914+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:41:05.059914+0000","last_undegraded":"2023-10-06T01:41:05.059914+0000","last_fullsized":"2023-10-06T01:41:05.059914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062784+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:41:05.062784+0000","last_peered":"2023-10-06T01:41:05.062784+0000","last_clean":"2023-10-06T01:41:05.062784+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:41:05.062784+0000","last_undegraded":"2023-10-06T01:41:05.062784+0000","last_fullsized":"2023-10-06T01:41:05.062784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067606+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:41:05.067606+0000","last_peered":"2023-10-06T01:41:05.067606+0000","last_clean":"2023-10-06T01:41:05.067606+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:41:05.067606+0000","last_undegraded":"2023-10-06T01:41:05.067606+0000","last_fullsized":"2023-10-06T01:41:05.067606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041167+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:56.041167+0000","last_peered":"2023-10-06T01:40:56.041167+0000","last_clean":"2023-10-06T01:40:56.041167+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:56.041167+0000","last_undegraded":"2023-10-06T01:40:56.041167+0000","last_fullsized":"2023-10-06T01:40:56.041167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063049+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:41:05.063049+0000","last_peered":"2023-10-06T01:41:05.063049+0000","last_clean":"2023-10-06T01:41:05.063049+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:41:05.063049+0000","last_undegraded":"2023-10-06T01:41:05.063049+0000","last_fullsized":"2023-10-06T01:41:05.063049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"36'12","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066771+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:41:05.066771+0000","last_peered":"2023-10-06T01:41:05.066771+0000","last_clean":"2023-10-06T01:41:05.066771+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:41:05.066771+0000","last_undegraded":"2023-10-06T01:41:05.066771+0000","last_fullsized":"2023-10-06T01:41:05.066771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059829+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:41:05.059829+0000","last_peered":"2023-10-06T01:41:05.059829+0000","last_clean":"2023-10-06T01:41:05.059829+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:41:05.059829+0000","last_undegraded":"2023-10-06T01:41:05.059829+0000","last_fullsized":"2023-10-06T01:41:05.059829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074919+0000","last_change":"2023-10-06T01:41:05.072432+0000","last_active":"2023-10-06T01:41:05.074919+0000","last_peered":"2023-10-06T01:41:05.074919+0000","last_clean":"2023-10-06T01:41:05.074919+0000","last_became_active":"2023-10-06T01:41:05.072154+0000","last_became_peered":"2023-10-06T01:41:05.072154+0000","last_unstale":"2023-10-06T01:41:05.074919+0000","last_undegraded":"2023-10-06T01:41:05.074919+0000","last_fullsized":"2023-10-06T01:41:05.074919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:55.047887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063306+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:41:05.063306+0000","last_peered":"2023-10-06T01:41:05.063306+0000","last_clean":"2023-10-06T01:41:05.063306+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:41:05.063306+0000","last_undegraded":"2023-10-06T01:41:05.063306+0000","last_fullsized":"2023-10-06T01:41:05.063306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068615+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:41:05.068615+0000","last_peered":"2023-10-06T01:41:05.068615+0000","last_clean":"2023-10-06T01:41:05.068615+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:41:05.068615+0000","last_undegraded":"2023-10-06T01:41:05.068615+0000","last_fullsized":"2023-10-06T01:41:05.068615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041664+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:56.041664+0000","last_peered":"2023-10-06T01:40:56.041664+0000","last_clean":"2023-10-06T01:40:56.041664+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:56.041664+0000","last_undegraded":"2023-10-06T01:40:56.041664+0000","last_fullsized":"2023-10-06T01:40:56.041664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063096+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:41:05.063096+0000","last_peered":"2023-10-06T01:41:05.063096+0000","last_clean":"2023-10-06T01:41:05.063096+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:41:05.063096+0000","last_undegraded":"2023-10-06T01:41:05.063096+0000","last_fullsized":"2023-10-06T01:41:05.063096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075586+0000","last_change":"2023-10-06T01:41:05.072321+0000","last_active":"2023-10-06T01:41:05.075586+0000","last_peered":"2023-10-06T01:41:05.075586+0000","last_clean":"2023-10-06T01:41:05.075586+0000","last_became_active":"2023-10-06T01:41:05.072037+0000","last_became_peered":"2023-10-06T01:41:05.072037+0000","last_unstale":"2023-10-06T01:41:05.075586+0000","last_undegraded":"2023-10-06T01:41:05.075586+0000","last_fullsized":"2023-10-06T01:41:05.075586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067494+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:41:05.067494+0000","last_peered":"2023-10-06T01:41:05.067494+0000","last_clean":"2023-10-06T01:41:05.067494+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:41:05.067494+0000","last_undegraded":"2023-10-06T01:41:05.067494+0000","last_fullsized":"2023-10-06T01:41:05.067494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062861+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:41:05.062861+0000","last_peered":"2023-10-06T01:41:05.062861+0000","last_clean":"2023-10-06T01:41:05.062861+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:41:05.062861+0000","last_undegraded":"2023-10-06T01:41:05.062861+0000","last_fullsized":"2023-10-06T01:41:05.062861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062619+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:41:05.062619+0000","last_peered":"2023-10-06T01:41:05.062619+0000","last_clean":"2023-10-06T01:41:05.062619+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:41:05.062619+0000","last_undegraded":"2023-10-06T01:41:05.062619+0000","last_fullsized":"2023-10-06T01:41:05.062619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'3","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.514586+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:41:01.514586+0000","last_peered":"2023-10-06T01:41:01.514586+0000","last_clean":"2023-10-06T01:41:01.514586+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:41:01.514586+0000","last_undegraded":"2023-10-06T01:41:01.514586+0000","last_fullsized":"2023-10-06T01:41:01.514586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063152+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:41:05.063152+0000","last_peered":"2023-10-06T01:41:05.063152+0000","last_clean":"2023-10-06T01:41:05.063152+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:41:05.063152+0000","last_undegraded":"2023-10-06T01:41:05.063152+0000","last_fullsized":"2023-10-06T01:41:05.063152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067927+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:41:05.067927+0000","last_peered":"2023-10-06T01:41:05.067927+0000","last_clean":"2023-10-06T01:41:05.067927+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:41:05.067927+0000","last_undegraded":"2023-10-06T01:41:05.067927+0000","last_fullsized":"2023-10-06T01:41:05.067927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071094+0000","last_change":"2023-10-06T01:41:05.068576+0000","last_active":"2023-10-06T01:41:05.071094+0000","last_peered":"2023-10-06T01:41:05.071094+0000","last_clean":"2023-10-06T01:41:05.071094+0000","last_became_active":"2023-10-06T01:41:05.068329+0000","last_became_peered":"2023-10-06T01:41:05.068329+0000","last_unstale":"2023-10-06T01:41:05.071094+0000","last_undegraded":"2023-10-06T01:41:05.071094+0000","last_fullsized":"2023-10-06T01:41:05.071094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164999+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:56.164999+0000","last_peered":"2023-10-06T01:40:56.164999+0000","last_clean":"2023-10-06T01:40:56.164999+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:56.164999+0000","last_undegraded":"2023-10-06T01:40:56.164999+0000","last_fullsized":"2023-10-06T01:40:56.164999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063903+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:41:05.063903+0000","last_peered":"2023-10-06T01:41:05.063903+0000","last_clean":"2023-10-06T01:41:05.063903+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:41:05.063903+0000","last_undegraded":"2023-10-06T01:41:05.063903+0000","last_fullsized":"2023-10-06T01:41:05.063903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165119+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:56.165119+0000","last_peered":"2023-10-06T01:40:56.165119+0000","last_clean":"2023-10-06T01:40:56.165119+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:56.165119+0000","last_undegraded":"2023-10-06T01:40:56.165119+0000","last_fullsized":"2023-10-06T01:40:56.165119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066774+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:41:05.066774+0000","last_peered":"2023-10-06T01:41:05.066774+0000","last_clean":"2023-10-06T01:41:05.066774+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:41:05.066774+0000","last_undegraded":"2023-10-06T01:41:05.066774+0000","last_fullsized":"2023-10-06T01:41:05.066774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077743+0000","last_change":"2023-10-06T01:41:05.075317+0000","last_active":"2023-10-06T01:41:05.077743+0000","last_peered":"2023-10-06T01:41:05.077743+0000","last_clean":"2023-10-06T01:41:05.077743+0000","last_became_active":"2023-10-06T01:41:05.075139+0000","last_became_peered":"2023-10-06T01:41:05.075139+0000","last_unstale":"2023-10-06T01:41:05.077743+0000","last_undegraded":"2023-10-06T01:41:05.077743+0000","last_fullsized":"2023-10-06T01:41:05.077743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"36'10","reported_seq":24,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041054+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:56.041054+0000","last_peered":"2023-10-06T01:40:56.041054+0000","last_clean":"2023-10-06T01:40:56.041054+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:56.041054+0000","last_undegraded":"2023-10-06T01:40:56.041054+0000","last_fullsized":"2023-10-06T01:40:56.041054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165263+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:56.165263+0000","last_peered":"2023-10-06T01:40:56.165263+0000","last_clean":"2023-10-06T01:40:56.165263+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:56.165263+0000","last_undegraded":"2023-10-06T01:40:56.165263+0000","last_fullsized":"2023-10-06T01:40:56.165263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074604+0000","last_change":"2023-10-06T01:41:05.069137+0000","last_active":"2023-10-06T01:41:05.074604+0000","last_peered":"2023-10-06T01:41:05.074604+0000","last_clean":"2023-10-06T01:41:05.074604+0000","last_became_active":"2023-10-06T01:41:05.068861+0000","last_became_peered":"2023-10-06T01:41:05.068861+0000","last_unstale":"2023-10-06T01:41:05.074604+0000","last_undegraded":"2023-10-06T01:41:05.074604+0000","last_fullsized":"2023-10-06T01:41:05.074604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:54.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063296+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:41:05.063296+0000","last_peered":"2023-10-06T01:41:05.063296+0000","last_clean":"2023-10-06T01:41:05.063296+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:41:05.063296+0000","last_undegraded":"2023-10-06T01:41:05.063296+0000","last_fullsized":"2023-10-06T01:41:05.063296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063779+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:41:05.063779+0000","last_peered":"2023-10-06T01:41:05.063779+0000","last_clean":"2023-10-06T01:41:05.063779+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:41:05.063779+0000","last_undegraded":"2023-10-06T01:41:05.063779+0000","last_fullsized":"2023-10-06T01:41:05.063779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073651+0000","last_change":"2023-10-06T01:41:05.071706+0000","last_active":"2023-10-06T01:41:05.073651+0000","last_peered":"2023-10-06T01:41:05.073651+0000","last_clean":"2023-10-06T01:41:05.073651+0000","last_became_active":"2023-10-06T01:41:05.071432+0000","last_became_peered":"2023-10-06T01:41:05.071432+0000","last_unstale":"2023-10-06T01:41:05.073651+0000","last_undegraded":"2023-10-06T01:41:05.073651+0000","last_fullsized":"2023-10-06T01:41:05.073651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164781+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:56.164781+0000","last_peered":"2023-10-06T01:40:56.164781+0000","last_clean":"2023-10-06T01:40:56.164781+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:56.164781+0000","last_undegraded":"2023-10-06T01:40:56.164781+0000","last_fullsized":"2023-10-06T01:40:56.164781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165632+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:56.165632+0000","last_peered":"2023-10-06T01:40:56.165632+0000","last_clean":"2023-10-06T01:40:56.165632+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:56.165632+0000","last_undegraded":"2023-10-06T01:40:56.165632+0000","last_fullsized":"2023-10-06T01:40:56.165632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070176+0000","last_change":"2023-10-06T01:41:05.066490+0000","last_active":"2023-10-06T01:41:05.070176+0000","last_peered":"2023-10-06T01:41:05.070176+0000","last_clean":"2023-10-06T01:41:05.070176+0000","last_became_active":"2023-10-06T01:41:05.065985+0000","last_became_peered":"2023-10-06T01:41:05.065985+0000","last_unstale":"2023-10-06T01:41:05.070176+0000","last_undegraded":"2023-10-06T01:41:05.070176+0000","last_fullsized":"2023-10-06T01:41:05.070176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067405+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:41:05.067405+0000","last_peered":"2023-10-06T01:41:05.067405+0000","last_clean":"2023-10-06T01:41:05.067405+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:41:05.067405+0000","last_undegraded":"2023-10-06T01:41:05.067405+0000","last_fullsized":"2023-10-06T01:41:05.067405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'5","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.515239+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:41:01.515239+0000","last_peered":"2023-10-06T01:41:01.515239+0000","last_clean":"2023-10-06T01:41:01.515239+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:41:01.515239+0000","last_undegraded":"2023-10-06T01:41:01.515239+0000","last_fullsized":"2023-10-06T01:41:01.515239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164687+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:56.164687+0000","last_peered":"2023-10-06T01:40:56.164687+0000","last_clean":"2023-10-06T01:40:56.164687+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:56.164687+0000","last_undegraded":"2023-10-06T01:40:56.164687+0000","last_fullsized":"2023-10-06T01:40:56.164687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'22","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060056+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:41:05.060056+0000","last_peered":"2023-10-06T01:41:05.060056+0000","last_clean":"2023-10-06T01:41:05.060056+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:41:05.060056+0000","last_undegraded":"2023-10-06T01:41:05.060056+0000","last_fullsized":"2023-10-06T01:41:05.060056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"36'10","reported_seq":24,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041082+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:56.041082+0000","last_peered":"2023-10-06T01:40:56.041082+0000","last_clean":"2023-10-06T01:40:56.041082+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:56.041082+0000","last_undegraded":"2023-10-06T01:40:56.041082+0000","last_fullsized":"2023-10-06T01:40:56.041082+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067767+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:41:05.067767+0000","last_peered":"2023-10-06T01:41:05.067767+0000","last_clean":"2023-10-06T01:41:05.067767+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:41:05.067767+0000","last_undegraded":"2023-10-06T01:41:05.067767+0000","last_fullsized":"2023-10-06T01:41:05.067767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074931+0000","last_change":"2023-10-06T01:41:05.072492+0000","last_active":"2023-10-06T01:41:05.074931+0000","last_peered":"2023-10-06T01:41:05.074931+0000","last_clean":"2023-10-06T01:41:05.074931+0000","last_became_active":"2023-10-06T01:41:05.072225+0000","last_became_peered":"2023-10-06T01:41:05.072225+0000","last_unstale":"2023-10-06T01:41:05.074931+0000","last_undegraded":"2023-10-06T01:41:05.074931+0000","last_fullsized":"2023-10-06T01:41:05.074931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.539127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"36'15","reported_seq":27,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165053+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:56.165053+0000","last_peered":"2023-10-06T01:40:56.165053+0000","last_clean":"2023-10-06T01:40:56.165053+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:56.165053+0000","last_undegraded":"2023-10-06T01:40:56.165053+0000","last_fullsized":"2023-10-06T01:40:56.165053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164653+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:56.164653+0000","last_peered":"2023-10-06T01:40:56.164653+0000","last_clean":"2023-10-06T01:40:56.164653+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:56.164653+0000","last_undegraded":"2023-10-06T01:40:56.164653+0000","last_fullsized":"2023-10-06T01:40:56.164653+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067386+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:41:05.067386+0000","last_peered":"2023-10-06T01:41:05.067386+0000","last_clean":"2023-10-06T01:41:05.067386+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:41:05.067386+0000","last_undegraded":"2023-10-06T01:41:05.067386+0000","last_fullsized":"2023-10-06T01:41:05.067386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066926+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:41:05.066926+0000","last_peered":"2023-10-06T01:41:05.066926+0000","last_clean":"2023-10-06T01:41:05.066926+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:41:05.066926+0000","last_undegraded":"2023-10-06T01:41:05.066926+0000","last_fullsized":"2023-10-06T01:41:05.066926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073491+0000","last_change":"2023-10-06T01:41:05.070833+0000","last_active":"2023-10-06T01:41:05.073491+0000","last_peered":"2023-10-06T01:41:05.073491+0000","last_clean":"2023-10-06T01:41:05.073491+0000","last_became_active":"2023-10-06T01:41:05.070486+0000","last_became_peered":"2023-10-06T01:41:05.070486+0000","last_unstale":"2023-10-06T01:41:05.073491+0000","last_undegraded":"2023-10-06T01:41:05.073491+0000","last_fullsized":"2023-10-06T01:41:05.073491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164917+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:56.164917+0000","last_peered":"2023-10-06T01:40:56.164917+0000","last_clean":"2023-10-06T01:40:56.164917+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:56.164917+0000","last_undegraded":"2023-10-06T01:40:56.164917+0000","last_fullsized":"2023-10-06T01:40:56.164917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'15","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.519887+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:41:01.519887+0000","last_peered":"2023-10-06T01:41:01.519887+0000","last_clean":"2023-10-06T01:41:01.519887+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:41:01.519887+0000","last_undegraded":"2023-10-06T01:41:01.519887+0000","last_fullsized":"2023-10-06T01:41:01.519887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071482+0000","last_change":"2023-10-06T01:41:05.069668+0000","last_active":"2023-10-06T01:41:05.071482+0000","last_peered":"2023-10-06T01:41:05.071482+0000","last_clean":"2023-10-06T01:41:05.071482+0000","last_became_active":"2023-10-06T01:41:05.069033+0000","last_became_peered":"2023-10-06T01:41:05.069033+0000","last_unstale":"2023-10-06T01:41:05.071482+0000","last_undegraded":"2023-10-06T01:41:05.071482+0000","last_fullsized":"2023-10-06T01:41:05.071482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:14:32.941476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059936+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:41:05.059936+0000","last_peered":"2023-10-06T01:41:05.059936+0000","last_clean":"2023-10-06T01:41:05.059936+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:41:05.059936+0000","last_undegraded":"2023-10-06T01:41:05.059936+0000","last_fullsized":"2023-10-06T01:41:05.059936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060008+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:41:05.060008+0000","last_peered":"2023-10-06T01:41:05.060008+0000","last_clean":"2023-10-06T01:41:05.060008+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:41:05.060008+0000","last_undegraded":"2023-10-06T01:41:05.060008+0000","last_fullsized":"2023-10-06T01:41:05.060008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075650+0000","last_change":"2023-10-06T01:41:05.072868+0000","last_active":"2023-10-06T01:41:05.075650+0000","last_peered":"2023-10-06T01:41:05.075650+0000","last_clean":"2023-10-06T01:41:05.075650+0000","last_became_active":"2023-10-06T01:41:05.072621+0000","last_became_peered":"2023-10-06T01:41:05.072621+0000","last_unstale":"2023-10-06T01:41:05.075650+0000","last_undegraded":"2023-10-06T01:41:05.075650+0000","last_fullsized":"2023-10-06T01:41:05.075650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165859+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:56.165859+0000","last_peered":"2023-10-06T01:40:56.165859+0000","last_clean":"2023-10-06T01:40:56.165859+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:56.165859+0000","last_undegraded":"2023-10-06T01:40:56.165859+0000","last_fullsized":"2023-10-06T01:40:56.165859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'3","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063389+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:41:05.063389+0000","last_peered":"2023-10-06T01:41:05.063389+0000","last_clean":"2023-10-06T01:41:05.063389+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:41:05.063389+0000","last_undegraded":"2023-10-06T01:41:05.063389+0000","last_fullsized":"2023-10-06T01:41:05.063389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.080676+0000","last_change":"2023-10-06T01:41:05.073304+0000","last_active":"2023-10-06T01:41:05.080676+0000","last_peered":"2023-10-06T01:41:05.080676+0000","last_clean":"2023-10-06T01:41:05.080676+0000","last_became_active":"2023-10-06T01:41:05.073056+0000","last_became_peered":"2023-10-06T01:41:05.073056+0000","last_unstale":"2023-10-06T01:41:05.080676+0000","last_undegraded":"2023-10-06T01:41:05.080676+0000","last_fullsized":"2023-10-06T01:41:05.080676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:14.536410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041175+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:56.041175+0000","last_peered":"2023-10-06T01:40:56.041175+0000","last_clean":"2023-10-06T01:40:56.041175+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:56.041175+0000","last_undegraded":"2023-10-06T01:40:56.041175+0000","last_fullsized":"2023-10-06T01:40:56.041175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'2","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067860+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:41:05.067860+0000","last_peered":"2023-10-06T01:41:05.067860+0000","last_clean":"2023-10-06T01:41:05.067860+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:41:05.067860+0000","last_undegraded":"2023-10-06T01:41:05.067860+0000","last_fullsized":"2023-10-06T01:41:05.067860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068813+0000","last_change":"2023-10-06T01:41:05.066269+0000","last_active":"2023-10-06T01:41:05.068813+0000","last_peered":"2023-10-06T01:41:05.068813+0000","last_clean":"2023-10-06T01:41:05.068813+0000","last_became_active":"2023-10-06T01:41:05.065979+0000","last_became_peered":"2023-10-06T01:41:05.065979+0000","last_unstale":"2023-10-06T01:41:05.068813+0000","last_undegraded":"2023-10-06T01:41:05.068813+0000","last_fullsized":"2023-10-06T01:41:05.068813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041494+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:56.041494+0000","last_peered":"2023-10-06T01:40:56.041494+0000","last_clean":"2023-10-06T01:40:56.041494+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:56.041494+0000","last_undegraded":"2023-10-06T01:40:56.041494+0000","last_fullsized":"2023-10-06T01:40:56.041494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067544+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:41:05.067544+0000","last_peered":"2023-10-06T01:41:05.067544+0000","last_clean":"2023-10-06T01:41:05.067544+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:41:05.067544+0000","last_undegraded":"2023-10-06T01:41:05.067544+0000","last_fullsized":"2023-10-06T01:41:05.067544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067040+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:41:05.067040+0000","last_peered":"2023-10-06T01:41:05.067040+0000","last_clean":"2023-10-06T01:41:05.067040+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:41:05.067040+0000","last_undegraded":"2023-10-06T01:41:05.067040+0000","last_fullsized":"2023-10-06T01:41:05.067040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'4","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067848+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:41:05.067848+0000","last_peered":"2023-10-06T01:41:05.067848+0000","last_clean":"2023-10-06T01:41:05.067848+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:41:05.067848+0000","last_undegraded":"2023-10-06T01:41:05.067848+0000","last_fullsized":"2023-10-06T01:41:05.067848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074693+0000","last_change":"2023-10-06T01:41:05.069995+0000","last_active":"2023-10-06T01:41:05.074693+0000","last_peered":"2023-10-06T01:41:05.074693+0000","last_clean":"2023-10-06T01:41:05.074693+0000","last_became_active":"2023-10-06T01:41:05.069709+0000","last_became_peered":"2023-10-06T01:41:05.069709+0000","last_unstale":"2023-10-06T01:41:05.074693+0000","last_undegraded":"2023-10-06T01:41:05.074693+0000","last_fullsized":"2023-10-06T01:41:05.074693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:50.555907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062679+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:41:05.062679+0000","last_peered":"2023-10-06T01:41:05.062679+0000","last_clean":"2023-10-06T01:41:05.062679+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:41:05.062679+0000","last_undegraded":"2023-10-06T01:41:05.062679+0000","last_fullsized":"2023-10-06T01:41:05.062679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066603+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:41:05.066603+0000","last_peered":"2023-10-06T01:41:05.066603+0000","last_clean":"2023-10-06T01:41:05.066603+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:41:05.066603+0000","last_undegraded":"2023-10-06T01:41:05.066603+0000","last_fullsized":"2023-10-06T01:41:05.066603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077363+0000","last_change":"2023-10-06T01:41:05.071858+0000","last_active":"2023-10-06T01:41:05.077363+0000","last_peered":"2023-10-06T01:41:05.077363+0000","last_clean":"2023-10-06T01:41:05.077363+0000","last_became_active":"2023-10-06T01:41:05.071590+0000","last_became_peered":"2023-10-06T01:41:05.071590+0000","last_unstale":"2023-10-06T01:41:05.077363+0000","last_undegraded":"2023-10-06T01:41:05.077363+0000","last_fullsized":"2023-10-06T01:41:05.077363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:39.551110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059831+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:41:05.059831+0000","last_peered":"2023-10-06T01:41:05.059831+0000","last_clean":"2023-10-06T01:41:05.059831+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:41:05.059831+0000","last_undegraded":"2023-10-06T01:41:05.059831+0000","last_fullsized":"2023-10-06T01:41:05.059831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164866+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:56.164866+0000","last_peered":"2023-10-06T01:40:56.164866+0000","last_clean":"2023-10-06T01:40:56.164866+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:56.164866+0000","last_undegraded":"2023-10-06T01:40:56.164866+0000","last_fullsized":"2023-10-06T01:40:56.164866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"36'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164922+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:56.164922+0000","last_peered":"2023-10-06T01:40:56.164922+0000","last_clean":"2023-10-06T01:40:56.164922+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:56.164922+0000","last_undegraded":"2023-10-06T01:40:56.164922+0000","last_fullsized":"2023-10-06T01:40:56.164922+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164422+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:56.164422+0000","last_peered":"2023-10-06T01:40:56.164422+0000","last_clean":"2023-10-06T01:40:56.164422+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:56.164422+0000","last_undegraded":"2023-10-06T01:40:56.164422+0000","last_fullsized":"2023-10-06T01:40:56.164422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067681+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:41:05.067681+0000","last_peered":"2023-10-06T01:41:05.067681+0000","last_clean":"2023-10-06T01:41:05.067681+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:41:05.067681+0000","last_undegraded":"2023-10-06T01:41:05.067681+0000","last_fullsized":"2023-10-06T01:41:05.067681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063996+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:41:05.063996+0000","last_peered":"2023-10-06T01:41:05.063996+0000","last_clean":"2023-10-06T01:41:05.063996+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:41:05.063996+0000","last_undegraded":"2023-10-06T01:41:05.063996+0000","last_fullsized":"2023-10-06T01:41:05.063996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062842+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:05.062842+0000","last_peered":"2023-10-06T01:41:05.062842+0000","last_clean":"2023-10-06T01:41:05.062842+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:05.062842+0000","last_undegraded":"2023-10-06T01:41:05.062842+0000","last_fullsized":"2023-10-06T01:41:05.062842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074573+0000","last_change":"2023-10-06T01:41:05.070827+0000","last_active":"2023-10-06T01:41:05.074573+0000","last_peered":"2023-10-06T01:41:05.074573+0000","last_clean":"2023-10-06T01:41:05.074573+0000","last_became_active":"2023-10-06T01:41:05.070649+0000","last_became_peered":"2023-10-06T01:41:05.070649+0000","last_unstale":"2023-10-06T01:41:05.074573+0000","last_undegraded":"2023-10-06T01:41:05.074573+0000","last_fullsized":"2023-10-06T01:41:05.074573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:26.468333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067196+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:41:05.067196+0000","last_peered":"2023-10-06T01:41:05.067196+0000","last_clean":"2023-10-06T01:41:05.067196+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:41:05.067196+0000","last_undegraded":"2023-10-06T01:41:05.067196+0000","last_fullsized":"2023-10-06T01:41:05.067196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165177+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:56.165177+0000","last_peered":"2023-10-06T01:40:56.165177+0000","last_clean":"2023-10-06T01:40:56.165177+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:56.165177+0000","last_undegraded":"2023-10-06T01:40:56.165177+0000","last_fullsized":"2023-10-06T01:40:56.165177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067611+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:05.067611+0000","last_peered":"2023-10-06T01:41:05.067611+0000","last_clean":"2023-10-06T01:41:05.067611+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:05.067611+0000","last_undegraded":"2023-10-06T01:41:05.067611+0000","last_fullsized":"2023-10-06T01:41:05.067611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071384+0000","last_change":"2023-10-06T01:41:05.068635+0000","last_active":"2023-10-06T01:41:05.071384+0000","last_peered":"2023-10-06T01:41:05.071384+0000","last_clean":"2023-10-06T01:41:05.071384+0000","last_became_active":"2023-10-06T01:41:05.068364+0000","last_became_peered":"2023-10-06T01:41:05.068364+0000","last_unstale":"2023-10-06T01:41:05.071384+0000","last_undegraded":"2023-10-06T01:41:05.071384+0000","last_fullsized":"2023-10-06T01:41:05.071384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:53.027266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"36'5","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164843+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:56.164843+0000","last_peered":"2023-10-06T01:40:56.164843+0000","last_clean":"2023-10-06T01:40:56.164843+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:56.164843+0000","last_undegraded":"2023-10-06T01:40:56.164843+0000","last_fullsized":"2023-10-06T01:40:56.164843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062581+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:41:05.062581+0000","last_peered":"2023-10-06T01:41:05.062581+0000","last_clean":"2023-10-06T01:41:05.062581+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:41:05.062581+0000","last_undegraded":"2023-10-06T01:41:05.062581+0000","last_fullsized":"2023-10-06T01:41:05.062581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":178,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164776+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:56.164776+0000","last_peered":"2023-10-06T01:40:56.164776+0000","last_clean":"2023-10-06T01:40:56.164776+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:56.164776+0000","last_undegraded":"2023-10-06T01:40:56.164776+0000","last_fullsized":"2023-10-06T01:40:56.164776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075630+0000","last_change":"2023-10-06T01:41:05.072823+0000","last_active":"2023-10-06T01:41:05.075630+0000","last_peered":"2023-10-06T01:41:05.075630+0000","last_clean":"2023-10-06T01:41:05.075630+0000","last_became_active":"2023-10-06T01:41:05.072511+0000","last_became_peered":"2023-10-06T01:41:05.072511+0000","last_unstale":"2023-10-06T01:41:05.075630+0000","last_undegraded":"2023-10-06T01:41:05.075630+0000","last_fullsized":"2023-10-06T01:41:05.075630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165361+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:56.165361+0000","last_peered":"2023-10-06T01:40:56.165361+0000","last_clean":"2023-10-06T01:40:56.165361+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:56.165361+0000","last_undegraded":"2023-10-06T01:40:56.165361+0000","last_fullsized":"2023-10-06T01:40:56.165361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164979+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:56.164979+0000","last_peered":"2023-10-06T01:40:56.164979+0000","last_clean":"2023-10-06T01:40:56.164979+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:56.164979+0000","last_undegraded":"2023-10-06T01:40:56.164979+0000","last_fullsized":"2023-10-06T01:40:56.164979+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164838+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:56.164838+0000","last_peered":"2023-10-06T01:40:56.164838+0000","last_clean":"2023-10-06T01:40:56.164838+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:56.164838+0000","last_undegraded":"2023-10-06T01:40:56.164838+0000","last_fullsized":"2023-10-06T01:40:56.164838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"36'7","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165559+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:56.165559+0000","last_peered":"2023-10-06T01:40:56.165559+0000","last_clean":"2023-10-06T01:40:56.165559+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:56.165559+0000","last_undegraded":"2023-10-06T01:40:56.165559+0000","last_fullsized":"2023-10-06T01:40:56.165559+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068333+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:41:05.068333+0000","last_peered":"2023-10-06T01:41:05.068333+0000","last_clean":"2023-10-06T01:41:05.068333+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:41:05.068333+0000","last_undegraded":"2023-10-06T01:41:05.068333+0000","last_fullsized":"2023-10-06T01:41:05.068333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164836+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:56.164836+0000","last_peered":"2023-10-06T01:40:56.164836+0000","last_clean":"2023-10-06T01:40:56.164836+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:56.164836+0000","last_undegraded":"2023-10-06T01:40:56.164836+0000","last_fullsized":"2023-10-06T01:40:56.164836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074585+0000","last_change":"2023-10-06T01:41:05.071521+0000","last_active":"2023-10-06T01:41:05.074585+0000","last_peered":"2023-10-06T01:41:05.074585+0000","last_clean":"2023-10-06T01:41:05.074585+0000","last_became_active":"2023-10-06T01:41:05.071272+0000","last_became_peered":"2023-10-06T01:41:05.071272+0000","last_unstale":"2023-10-06T01:41:05.074585+0000","last_undegraded":"2023-10-06T01:41:05.074585+0000","last_fullsized":"2023-10-06T01:41:05.074585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.431329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"36'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041758+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:56.041758+0000","last_peered":"2023-10-06T01:40:56.041758+0000","last_clean":"2023-10-06T01:40:56.041758+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:56.041758+0000","last_undegraded":"2023-10-06T01:40:56.041758+0000","last_fullsized":"2023-10-06T01:40:56.041758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'3","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068093+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:41:05.068093+0000","last_peered":"2023-10-06T01:41:05.068093+0000","last_clean":"2023-10-06T01:41:05.068093+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:41:05.068093+0000","last_undegraded":"2023-10-06T01:41:05.068093+0000","last_fullsized":"2023-10-06T01:41:05.068093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066856+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:05.066856+0000","last_peered":"2023-10-06T01:41:05.066856+0000","last_clean":"2023-10-06T01:41:05.066856+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:05.066856+0000","last_undegraded":"2023-10-06T01:41:05.066856+0000","last_fullsized":"2023-10-06T01:41:05.066856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077722+0000","last_change":"2023-10-06T01:41:05.073694+0000","last_active":"2023-10-06T01:41:05.077722+0000","last_peered":"2023-10-06T01:41:05.077722+0000","last_clean":"2023-10-06T01:41:05.077722+0000","last_became_active":"2023-10-06T01:41:05.073261+0000","last_became_peered":"2023-10-06T01:41:05.073261+0000","last_unstale":"2023-10-06T01:41:05.077722+0000","last_undegraded":"2023-10-06T01:41:05.077722+0000","last_fullsized":"2023-10-06T01:41:05.077722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066331+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:41:05.066331+0000","last_peered":"2023-10-06T01:41:05.066331+0000","last_clean":"2023-10-06T01:41:05.066331+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:41:05.066331+0000","last_undegraded":"2023-10-06T01:41:05.066331+0000","last_fullsized":"2023-10-06T01:41:05.066331+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062956+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:41:05.062956+0000","last_peered":"2023-10-06T01:41:05.062956+0000","last_clean":"2023-10-06T01:41:05.062956+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:41:05.062956+0000","last_undegraded":"2023-10-06T01:41:05.062956+0000","last_fullsized":"2023-10-06T01:41:05.062956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041323+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:56.041323+0000","last_peered":"2023-10-06T01:40:56.041323+0000","last_clean":"2023-10-06T01:40:56.041323+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:56.041323+0000","last_undegraded":"2023-10-06T01:40:56.041323+0000","last_fullsized":"2023-10-06T01:40:56.041323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"36'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062666+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:41:05.062666+0000","last_peered":"2023-10-06T01:41:05.062666+0000","last_clean":"2023-10-06T01:41:05.062666+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:41:05.062666+0000","last_undegraded":"2023-10-06T01:41:05.062666+0000","last_fullsized":"2023-10-06T01:41:05.062666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165539+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:56.165539+0000","last_peered":"2023-10-06T01:40:56.165539+0000","last_clean":"2023-10-06T01:40:56.165539+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:56.165539+0000","last_undegraded":"2023-10-06T01:40:56.165539+0000","last_fullsized":"2023-10-06T01:40:56.165539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041835+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:56.041835+0000","last_peered":"2023-10-06T01:40:56.041835+0000","last_clean":"2023-10-06T01:40:56.041835+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:56.041835+0000","last_undegraded":"2023-10-06T01:40:56.041835+0000","last_fullsized":"2023-10-06T01:40:56.041835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":40,"acting":40,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298332,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073508,"statfs":{"total":96636764160,"available":96331272192,"internally_reserved":0,"allocated":1069056,"data_stored":806541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.88},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4669440,"data_stored":4410477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.498}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.309}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4669440,"data_stored":4410477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51300000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":237748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.312}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":239101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41199999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.622},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:09.125 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:09.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.269+0000 7efcfd880700 1 Processor -- start 2023-10-06T01:41:09.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.269+0000 7efcfd880700 1 -- start start 2023-10-06T01:41:09.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcfd880700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efcf80b49e0 0x7efcf80aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcfd880700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efcf80b07a0 0x7efcf80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcfd880700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80b0ca0 con 0x7efcf81595a0 2023-10-06T01:41:09.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcfd880700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b0f80 con 0x7efcf80b49e0 2023-10-06T01:41:09.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcfd880700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b1260 con 0x7efcf80b07a0 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcee7fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efcf80b49e0 0x7efcf80aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcf6ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efcf81595a0 0x7efcf80aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55332/0 (socket says 172.21.15.98:55332) 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcf6ffd700 1 -- 172.21.15.98:0/2163792935 learned_addr learned my addr 172.21.15.98:0/2163792935 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcee7fc700 1 -- 172.21.15.98:0/2163792935 >> v2:172.21.15.184:3300/0 conn(0x7efcf80b07a0 msgr2=0x7efcf80b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcf77fe700 1 --2- 172.21.15.98:0/2163792935 >> v2:172.21.15.184:3300/0 conn(0x7efcf80b07a0 0x7efcf80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcee7fc700 1 --2- 172.21.15.98:0/2163792935 >> v2:172.21.15.184:3300/0 conn(0x7efcf80b07a0 0x7efcf80b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcf67fc700 1 -- 172.21.15.98:0/2163792935 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840953989 0 0) 0x7efcf80b0ca0 con 0x7efcf81595a0 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcee7fc700 1 -- 172.21.15.98:0/2163792935 >> v1:172.21.15.98:6789/0 conn(0x7efcf81595a0 legacy=0x7efcf80aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.270+0000 7efcee7fc700 1 -- 172.21.15.98:0/2163792935 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80b1540 con 0x7efcf80b49e0 2023-10-06T01:41:09.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcee7fc700 1 --2- 172.21.15.98:0/2163792935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf80aac80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efce00098e0 tx=0x7efce00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb044b58c1cf1fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcf67fc700 1 -- 172.21.15.98:0/2163792935 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0009e80 con 0x7efcf80b49e0 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcf67fc700 1 -- 172.21.15.98:0/2163792935 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce000b820 con 0x7efcf80b49e0 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcf67fc700 1 -- 172.21.15.98:0/2163792935 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce000bba0 con 0x7efcf80b49e0 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 -- 172.21.15.98:0/2163792935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 msgr2=0x7efcf80aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 --2- 172.21.15.98:0/2163792935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf80aac80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efce00098e0 tx=0x7efce00091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 -- 172.21.15.98:0/2163792935 shutdown_connections 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 --2- 172.21.15.98:0/2163792935 >> v2:172.21.15.184:3300/0 conn(0x7efcf80b07a0 0x7efcf80b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 --2- 172.21.15.98:0/2163792935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf80aac80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 -- 172.21.15.98:0/2163792935 >> 172.21.15.98:0/2163792935 conn(0x7efcf8099830 msgr2=0x7efcf806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 -- 172.21.15.98:0/2163792935 shutdown_connections 2023-10-06T01:41:09.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.271+0000 7efcfd880700 1 -- 172.21.15.98:0/2163792935 wait complete. 2023-10-06T01:41:09.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.272+0000 7efcfd880700 1 Processor -- start 2023-10-06T01:41:09.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 -- start start 2023-10-06T01:41:09.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf80b07a0 0x7efcf81569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf81595a0 0x7efcf8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815b6c0 con 0x7efcf80b49e0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815c200 con 0x7efcf81595a0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcfd880700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815b8d0 con 0x7efcf80b07a0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55136/0 (socket says 172.21.15.98:55136) 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcee7fc700 1 -- 172.21.15.98:0/247639272 learned_addr learned my addr 172.21.15.98:0/247639272 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcf6ffd700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf80b07a0 0x7efcf81569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.273+0000 7efcf77fe700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf81595a0 0x7efcf8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 -- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf80b07a0 msgr2=0x7efcf81569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf80b07a0 0x7efcf81569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 -- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf81595a0 msgr2=0x7efcf8158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf81595a0 0x7efcf8158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 -- 172.21.15.98:0/247639272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0009420 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcee7fc700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efce0013000 tx=0x7efce0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce000bba0 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0011420 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf815bbb0 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0011800 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.274+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf8156ef0 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.275+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf8157140 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.275+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efce0011b70 con 0x7efcf80b49e0 2023-10-06T01:41:09.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.275+0000 7efcef7fe700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 0x7efcd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.276+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7efce00864b0 con 0x7efcf80b49e0 2023-10-06T01:41:09.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.276+0000 7efcf6ffd700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 0x7efcd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.276+0000 7efcf6ffd700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 0x7efcd80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efce8002930 tx=0x7efce8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.280+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efce0053260 con 0x7efcf80b49e0 2023-10-06T01:41:09.458 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.457+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcf815c370 con 0x7efcd8066f30 2023-10-06T01:41:09.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.467+0000 7efcef7fe700 1 -- 172.21.15.98:0/247639272 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+390293 (secure 0 0 0) 0x7efcf815c370 con 0x7efcd8066f30 2023-10-06T01:41:09.472 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 msgr2=0x7efcd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 0x7efcd80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efce8002930 tx=0x7efce8009040 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 msgr2=0x7efcf81697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efce0013000 tx=0x7efce0017000 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 shutdown_connections 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efcd8066f30 0x7efcd80693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf81595a0 0x7efcf8158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf80b07a0 0x7efcf81569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.470+0000 7efcfd880700 1 --2- 172.21.15.98:0/247639272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf80b49e0 0x7efcf81697b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.471+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 >> 172.21.15.98:0/247639272 conn(0x7efcf8099830 msgr2=0x7efcf815d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.471+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 shutdown_connections 2023-10-06T01:41:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.471+0000 7efcfd880700 1 -- 172.21.15.98:0/247639272 wait complete. 2023-10-06T01:41:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:09.484 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-10-06T01:41:07.802543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":250,"acting":250,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":245,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395640,"kb_used_data":17336,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579080,"statfs":{"total":773094113280,"available":770640977920,"internally_reserved":0,"allocated":17752064,"data_stored":15606319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-90,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.752233"},"pg_stats":[{"pgid":"7.3f","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164681+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:40:56.164681+0000","last_peered":"2023-10-06T01:40:56.164681+0000","last_clean":"2023-10-06T01:40:56.164681+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:40:56.164681+0000","last_undegraded":"2023-10-06T01:40:56.164681+0000","last_fullsized":"2023-10-06T01:40:56.164681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068055+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:41:05.068055+0000","last_peered":"2023-10-06T01:41:05.068055+0000","last_clean":"2023-10-06T01:41:05.068055+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:41:05.068055+0000","last_undegraded":"2023-10-06T01:41:05.068055+0000","last_fullsized":"2023-10-06T01:41:05.068055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062731+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:41:05.062731+0000","last_peered":"2023-10-06T01:41:05.062731+0000","last_clean":"2023-10-06T01:41:05.062731+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:41:05.062731+0000","last_undegraded":"2023-10-06T01:41:05.062731+0000","last_fullsized":"2023-10-06T01:41:05.062731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165280+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:40:56.165280+0000","last_peered":"2023-10-06T01:40:56.165280+0000","last_clean":"2023-10-06T01:40:56.165280+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:40:56.165280+0000","last_undegraded":"2023-10-06T01:40:56.165280+0000","last_fullsized":"2023-10-06T01:40:56.165280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067188+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:41:05.067188+0000","last_peered":"2023-10-06T01:41:05.067188+0000","last_clean":"2023-10-06T01:41:05.067188+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:41:05.067188+0000","last_undegraded":"2023-10-06T01:41:05.067188+0000","last_fullsized":"2023-10-06T01:41:05.067188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067168+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:41:05.067168+0000","last_peered":"2023-10-06T01:41:05.067168+0000","last_clean":"2023-10-06T01:41:05.067168+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:41:05.067168+0000","last_undegraded":"2023-10-06T01:41:05.067168+0000","last_fullsized":"2023-10-06T01:41:05.067168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"36'5","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164751+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:40:56.164751+0000","last_peered":"2023-10-06T01:40:56.164751+0000","last_clean":"2023-10-06T01:40:56.164751+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:40:56.164751+0000","last_undegraded":"2023-10-06T01:40:56.164751+0000","last_fullsized":"2023-10-06T01:40:56.164751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164949+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:40:56.164949+0000","last_peered":"2023-10-06T01:40:56.164949+0000","last_clean":"2023-10-06T01:40:56.164949+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:40:56.164949+0000","last_undegraded":"2023-10-06T01:40:56.164949+0000","last_fullsized":"2023-10-06T01:40:56.164949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"36'11","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165851+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:40:56.165851+0000","last_peered":"2023-10-06T01:40:56.165851+0000","last_clean":"2023-10-06T01:40:56.165851+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:40:56.165851+0000","last_undegraded":"2023-10-06T01:40:56.165851+0000","last_fullsized":"2023-10-06T01:40:56.165851+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"36'15","reported_seq":27,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165484+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:40:56.165484+0000","last_peered":"2023-10-06T01:40:56.165484+0000","last_clean":"2023-10-06T01:40:56.165484+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:40:56.165484+0000","last_undegraded":"2023-10-06T01:40:56.165484+0000","last_fullsized":"2023-10-06T01:40:56.165484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165051+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:40:56.165051+0000","last_peered":"2023-10-06T01:40:56.165051+0000","last_clean":"2023-10-06T01:40:56.165051+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:40:56.165051+0000","last_undegraded":"2023-10-06T01:40:56.165051+0000","last_fullsized":"2023-10-06T01:40:56.165051+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066867+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:41:05.066867+0000","last_peered":"2023-10-06T01:41:05.066867+0000","last_clean":"2023-10-06T01:41:05.066867+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:41:05.066867+0000","last_undegraded":"2023-10-06T01:41:05.066867+0000","last_fullsized":"2023-10-06T01:41:05.066867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"36'10","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059981+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:41:05.059981+0000","last_peered":"2023-10-06T01:41:05.059981+0000","last_clean":"2023-10-06T01:41:05.059981+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:41:05.059981+0000","last_undegraded":"2023-10-06T01:41:05.059981+0000","last_fullsized":"2023-10-06T01:41:05.059981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165162+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:40:56.165162+0000","last_peered":"2023-10-06T01:40:56.165162+0000","last_clean":"2023-10-06T01:40:56.165162+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:40:56.165162+0000","last_undegraded":"2023-10-06T01:40:56.165162+0000","last_fullsized":"2023-10-06T01:40:56.165162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066663+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:41:05.066663+0000","last_peered":"2023-10-06T01:41:05.066663+0000","last_clean":"2023-10-06T01:41:05.066663+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:41:05.066663+0000","last_undegraded":"2023-10-06T01:41:05.066663+0000","last_fullsized":"2023-10-06T01:41:05.066663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"36'9","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040773+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:40:56.040773+0000","last_peered":"2023-10-06T01:40:56.040773+0000","last_clean":"2023-10-06T01:40:56.040773+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:40:56.040773+0000","last_undegraded":"2023-10-06T01:40:56.040773+0000","last_fullsized":"2023-10-06T01:40:56.040773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063591+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:41:05.063591+0000","last_peered":"2023-10-06T01:41:05.063591+0000","last_clean":"2023-10-06T01:41:05.063591+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:41:05.063591+0000","last_undegraded":"2023-10-06T01:41:05.063591+0000","last_fullsized":"2023-10-06T01:41:05.063591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062553+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:41:05.062553+0000","last_peered":"2023-10-06T01:41:05.062553+0000","last_clean":"2023-10-06T01:41:05.062553+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:41:05.062553+0000","last_undegraded":"2023-10-06T01:41:05.062553+0000","last_fullsized":"2023-10-06T01:41:05.062553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041241+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:40:56.041241+0000","last_peered":"2023-10-06T01:40:56.041241+0000","last_clean":"2023-10-06T01:40:56.041241+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:40:56.041241+0000","last_undegraded":"2023-10-06T01:40:56.041241+0000","last_fullsized":"2023-10-06T01:40:56.041241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059961+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:41:05.059961+0000","last_peered":"2023-10-06T01:41:05.059961+0000","last_clean":"2023-10-06T01:41:05.059961+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:41:05.059961+0000","last_undegraded":"2023-10-06T01:41:05.059961+0000","last_fullsized":"2023-10-06T01:41:05.059961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067584+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:41:05.067584+0000","last_peered":"2023-10-06T01:41:05.067584+0000","last_clean":"2023-10-06T01:41:05.067584+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:41:05.067584+0000","last_undegraded":"2023-10-06T01:41:05.067584+0000","last_fullsized":"2023-10-06T01:41:05.067584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165192+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:40:56.165192+0000","last_peered":"2023-10-06T01:40:56.165192+0000","last_clean":"2023-10-06T01:40:56.165192+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:40:56.165192+0000","last_undegraded":"2023-10-06T01:40:56.165192+0000","last_fullsized":"2023-10-06T01:40:56.165192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"36'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040905+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:40:56.040905+0000","last_peered":"2023-10-06T01:40:56.040905+0000","last_clean":"2023-10-06T01:40:56.040905+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:40:56.040905+0000","last_undegraded":"2023-10-06T01:40:56.040905+0000","last_fullsized":"2023-10-06T01:40:56.040905+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.040858+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:40:56.040858+0000","last_peered":"2023-10-06T01:40:56.040858+0000","last_clean":"2023-10-06T01:40:56.040858+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:40:56.040858+0000","last_undegraded":"2023-10-06T01:40:56.040858+0000","last_fullsized":"2023-10-06T01:40:56.040858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"36'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041744+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:40:56.041744+0000","last_peered":"2023-10-06T01:40:56.041744+0000","last_clean":"2023-10-06T01:40:56.041744+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:40:56.041744+0000","last_undegraded":"2023-10-06T01:40:56.041744+0000","last_fullsized":"2023-10-06T01:40:56.041744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"36'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059914+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:41:05.059914+0000","last_peered":"2023-10-06T01:41:05.059914+0000","last_clean":"2023-10-06T01:41:05.059914+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:41:05.059914+0000","last_undegraded":"2023-10-06T01:41:05.059914+0000","last_fullsized":"2023-10-06T01:41:05.059914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062784+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:41:05.062784+0000","last_peered":"2023-10-06T01:41:05.062784+0000","last_clean":"2023-10-06T01:41:05.062784+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:41:05.062784+0000","last_undegraded":"2023-10-06T01:41:05.062784+0000","last_fullsized":"2023-10-06T01:41:05.062784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067606+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:41:05.067606+0000","last_peered":"2023-10-06T01:41:05.067606+0000","last_clean":"2023-10-06T01:41:05.067606+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:41:05.067606+0000","last_undegraded":"2023-10-06T01:41:05.067606+0000","last_fullsized":"2023-10-06T01:41:05.067606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041167+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:40:56.041167+0000","last_peered":"2023-10-06T01:40:56.041167+0000","last_clean":"2023-10-06T01:40:56.041167+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:40:56.041167+0000","last_undegraded":"2023-10-06T01:40:56.041167+0000","last_fullsized":"2023-10-06T01:40:56.041167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063049+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:41:05.063049+0000","last_peered":"2023-10-06T01:41:05.063049+0000","last_clean":"2023-10-06T01:41:05.063049+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:41:05.063049+0000","last_undegraded":"2023-10-06T01:41:05.063049+0000","last_fullsized":"2023-10-06T01:41:05.063049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"36'12","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066771+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:41:05.066771+0000","last_peered":"2023-10-06T01:41:05.066771+0000","last_clean":"2023-10-06T01:41:05.066771+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:41:05.066771+0000","last_undegraded":"2023-10-06T01:41:05.066771+0000","last_fullsized":"2023-10-06T01:41:05.066771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059829+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:41:05.059829+0000","last_peered":"2023-10-06T01:41:05.059829+0000","last_clean":"2023-10-06T01:41:05.059829+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:41:05.059829+0000","last_undegraded":"2023-10-06T01:41:05.059829+0000","last_fullsized":"2023-10-06T01:41:05.059829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074919+0000","last_change":"2023-10-06T01:41:05.072432+0000","last_active":"2023-10-06T01:41:05.074919+0000","last_peered":"2023-10-06T01:41:05.074919+0000","last_clean":"2023-10-06T01:41:05.074919+0000","last_became_active":"2023-10-06T01:41:05.072154+0000","last_became_peered":"2023-10-06T01:41:05.072154+0000","last_unstale":"2023-10-06T01:41:05.074919+0000","last_undegraded":"2023-10-06T01:41:05.074919+0000","last_fullsized":"2023-10-06T01:41:05.074919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:55.047887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063306+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:41:05.063306+0000","last_peered":"2023-10-06T01:41:05.063306+0000","last_clean":"2023-10-06T01:41:05.063306+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:41:05.063306+0000","last_undegraded":"2023-10-06T01:41:05.063306+0000","last_fullsized":"2023-10-06T01:41:05.063306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068615+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:41:05.068615+0000","last_peered":"2023-10-06T01:41:05.068615+0000","last_clean":"2023-10-06T01:41:05.068615+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:41:05.068615+0000","last_undegraded":"2023-10-06T01:41:05.068615+0000","last_fullsized":"2023-10-06T01:41:05.068615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041664+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:40:56.041664+0000","last_peered":"2023-10-06T01:40:56.041664+0000","last_clean":"2023-10-06T01:40:56.041664+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:40:56.041664+0000","last_undegraded":"2023-10-06T01:40:56.041664+0000","last_fullsized":"2023-10-06T01:40:56.041664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063096+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:41:05.063096+0000","last_peered":"2023-10-06T01:41:05.063096+0000","last_clean":"2023-10-06T01:41:05.063096+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:41:05.063096+0000","last_undegraded":"2023-10-06T01:41:05.063096+0000","last_fullsized":"2023-10-06T01:41:05.063096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075586+0000","last_change":"2023-10-06T01:41:05.072321+0000","last_active":"2023-10-06T01:41:05.075586+0000","last_peered":"2023-10-06T01:41:05.075586+0000","last_clean":"2023-10-06T01:41:05.075586+0000","last_became_active":"2023-10-06T01:41:05.072037+0000","last_became_peered":"2023-10-06T01:41:05.072037+0000","last_unstale":"2023-10-06T01:41:05.075586+0000","last_undegraded":"2023-10-06T01:41:05.075586+0000","last_fullsized":"2023-10-06T01:41:05.075586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067494+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:41:05.067494+0000","last_peered":"2023-10-06T01:41:05.067494+0000","last_clean":"2023-10-06T01:41:05.067494+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:41:05.067494+0000","last_undegraded":"2023-10-06T01:41:05.067494+0000","last_fullsized":"2023-10-06T01:41:05.067494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062861+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:41:05.062861+0000","last_peered":"2023-10-06T01:41:05.062861+0000","last_clean":"2023-10-06T01:41:05.062861+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:41:05.062861+0000","last_undegraded":"2023-10-06T01:41:05.062861+0000","last_fullsized":"2023-10-06T01:41:05.062861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062619+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:41:05.062619+0000","last_peered":"2023-10-06T01:41:05.062619+0000","last_clean":"2023-10-06T01:41:05.062619+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:41:05.062619+0000","last_undegraded":"2023-10-06T01:41:05.062619+0000","last_fullsized":"2023-10-06T01:41:05.062619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'3","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.514586+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:41:01.514586+0000","last_peered":"2023-10-06T01:41:01.514586+0000","last_clean":"2023-10-06T01:41:01.514586+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:41:01.514586+0000","last_undegraded":"2023-10-06T01:41:01.514586+0000","last_fullsized":"2023-10-06T01:41:01.514586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063152+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:41:05.063152+0000","last_peered":"2023-10-06T01:41:05.063152+0000","last_clean":"2023-10-06T01:41:05.063152+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:41:05.063152+0000","last_undegraded":"2023-10-06T01:41:05.063152+0000","last_fullsized":"2023-10-06T01:41:05.063152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067927+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:41:05.067927+0000","last_peered":"2023-10-06T01:41:05.067927+0000","last_clean":"2023-10-06T01:41:05.067927+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:41:05.067927+0000","last_undegraded":"2023-10-06T01:41:05.067927+0000","last_fullsized":"2023-10-06T01:41:05.067927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071094+0000","last_change":"2023-10-06T01:41:05.068576+0000","last_active":"2023-10-06T01:41:05.071094+0000","last_peered":"2023-10-06T01:41:05.071094+0000","last_clean":"2023-10-06T01:41:05.071094+0000","last_became_active":"2023-10-06T01:41:05.068329+0000","last_became_peered":"2023-10-06T01:41:05.068329+0000","last_unstale":"2023-10-06T01:41:05.071094+0000","last_undegraded":"2023-10-06T01:41:05.071094+0000","last_fullsized":"2023-10-06T01:41:05.071094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164999+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:40:56.164999+0000","last_peered":"2023-10-06T01:40:56.164999+0000","last_clean":"2023-10-06T01:40:56.164999+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:40:56.164999+0000","last_undegraded":"2023-10-06T01:40:56.164999+0000","last_fullsized":"2023-10-06T01:40:56.164999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063903+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:41:05.063903+0000","last_peered":"2023-10-06T01:41:05.063903+0000","last_clean":"2023-10-06T01:41:05.063903+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:41:05.063903+0000","last_undegraded":"2023-10-06T01:41:05.063903+0000","last_fullsized":"2023-10-06T01:41:05.063903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"36'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165119+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:40:56.165119+0000","last_peered":"2023-10-06T01:40:56.165119+0000","last_clean":"2023-10-06T01:40:56.165119+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:40:56.165119+0000","last_undegraded":"2023-10-06T01:40:56.165119+0000","last_fullsized":"2023-10-06T01:40:56.165119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066774+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:41:05.066774+0000","last_peered":"2023-10-06T01:41:05.066774+0000","last_clean":"2023-10-06T01:41:05.066774+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:41:05.066774+0000","last_undegraded":"2023-10-06T01:41:05.066774+0000","last_fullsized":"2023-10-06T01:41:05.066774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077743+0000","last_change":"2023-10-06T01:41:05.075317+0000","last_active":"2023-10-06T01:41:05.077743+0000","last_peered":"2023-10-06T01:41:05.077743+0000","last_clean":"2023-10-06T01:41:05.077743+0000","last_became_active":"2023-10-06T01:41:05.075139+0000","last_became_peered":"2023-10-06T01:41:05.075139+0000","last_unstale":"2023-10-06T01:41:05.077743+0000","last_undegraded":"2023-10-06T01:41:05.077743+0000","last_fullsized":"2023-10-06T01:41:05.077743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"36'10","reported_seq":24,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041054+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:40:56.041054+0000","last_peered":"2023-10-06T01:40:56.041054+0000","last_clean":"2023-10-06T01:40:56.041054+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:40:56.041054+0000","last_undegraded":"2023-10-06T01:40:56.041054+0000","last_fullsized":"2023-10-06T01:40:56.041054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165263+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:40:56.165263+0000","last_peered":"2023-10-06T01:40:56.165263+0000","last_clean":"2023-10-06T01:40:56.165263+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:40:56.165263+0000","last_undegraded":"2023-10-06T01:40:56.165263+0000","last_fullsized":"2023-10-06T01:40:56.165263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074604+0000","last_change":"2023-10-06T01:41:05.069137+0000","last_active":"2023-10-06T01:41:05.074604+0000","last_peered":"2023-10-06T01:41:05.074604+0000","last_clean":"2023-10-06T01:41:05.074604+0000","last_became_active":"2023-10-06T01:41:05.068861+0000","last_became_peered":"2023-10-06T01:41:05.068861+0000","last_unstale":"2023-10-06T01:41:05.074604+0000","last_undegraded":"2023-10-06T01:41:05.074604+0000","last_fullsized":"2023-10-06T01:41:05.074604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:54.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063296+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:41:05.063296+0000","last_peered":"2023-10-06T01:41:05.063296+0000","last_clean":"2023-10-06T01:41:05.063296+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:41:05.063296+0000","last_undegraded":"2023-10-06T01:41:05.063296+0000","last_fullsized":"2023-10-06T01:41:05.063296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063779+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:41:05.063779+0000","last_peered":"2023-10-06T01:41:05.063779+0000","last_clean":"2023-10-06T01:41:05.063779+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:41:05.063779+0000","last_undegraded":"2023-10-06T01:41:05.063779+0000","last_fullsized":"2023-10-06T01:41:05.063779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073651+0000","last_change":"2023-10-06T01:41:05.071706+0000","last_active":"2023-10-06T01:41:05.073651+0000","last_peered":"2023-10-06T01:41:05.073651+0000","last_clean":"2023-10-06T01:41:05.073651+0000","last_became_active":"2023-10-06T01:41:05.071432+0000","last_became_peered":"2023-10-06T01:41:05.071432+0000","last_unstale":"2023-10-06T01:41:05.073651+0000","last_undegraded":"2023-10-06T01:41:05.073651+0000","last_fullsized":"2023-10-06T01:41:05.073651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164781+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:40:56.164781+0000","last_peered":"2023-10-06T01:40:56.164781+0000","last_clean":"2023-10-06T01:40:56.164781+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:40:56.164781+0000","last_undegraded":"2023-10-06T01:40:56.164781+0000","last_fullsized":"2023-10-06T01:40:56.164781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165632+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:40:56.165632+0000","last_peered":"2023-10-06T01:40:56.165632+0000","last_clean":"2023-10-06T01:40:56.165632+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:40:56.165632+0000","last_undegraded":"2023-10-06T01:40:56.165632+0000","last_fullsized":"2023-10-06T01:40:56.165632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070176+0000","last_change":"2023-10-06T01:41:05.066490+0000","last_active":"2023-10-06T01:41:05.070176+0000","last_peered":"2023-10-06T01:41:05.070176+0000","last_clean":"2023-10-06T01:41:05.070176+0000","last_became_active":"2023-10-06T01:41:05.065985+0000","last_became_peered":"2023-10-06T01:41:05.065985+0000","last_unstale":"2023-10-06T01:41:05.070176+0000","last_undegraded":"2023-10-06T01:41:05.070176+0000","last_fullsized":"2023-10-06T01:41:05.070176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067405+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:41:05.067405+0000","last_peered":"2023-10-06T01:41:05.067405+0000","last_clean":"2023-10-06T01:41:05.067405+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:41:05.067405+0000","last_undegraded":"2023-10-06T01:41:05.067405+0000","last_fullsized":"2023-10-06T01:41:05.067405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'5","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.515239+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:41:01.515239+0000","last_peered":"2023-10-06T01:41:01.515239+0000","last_clean":"2023-10-06T01:41:01.515239+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:41:01.515239+0000","last_undegraded":"2023-10-06T01:41:01.515239+0000","last_fullsized":"2023-10-06T01:41:01.515239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164687+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:40:56.164687+0000","last_peered":"2023-10-06T01:40:56.164687+0000","last_clean":"2023-10-06T01:40:56.164687+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:40:56.164687+0000","last_undegraded":"2023-10-06T01:40:56.164687+0000","last_fullsized":"2023-10-06T01:40:56.164687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'22","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060056+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:41:05.060056+0000","last_peered":"2023-10-06T01:41:05.060056+0000","last_clean":"2023-10-06T01:41:05.060056+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:41:05.060056+0000","last_undegraded":"2023-10-06T01:41:05.060056+0000","last_fullsized":"2023-10-06T01:41:05.060056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"36'10","reported_seq":24,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041082+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:40:56.041082+0000","last_peered":"2023-10-06T01:40:56.041082+0000","last_clean":"2023-10-06T01:40:56.041082+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:40:56.041082+0000","last_undegraded":"2023-10-06T01:40:56.041082+0000","last_fullsized":"2023-10-06T01:40:56.041082+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067767+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:41:05.067767+0000","last_peered":"2023-10-06T01:41:05.067767+0000","last_clean":"2023-10-06T01:41:05.067767+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:41:05.067767+0000","last_undegraded":"2023-10-06T01:41:05.067767+0000","last_fullsized":"2023-10-06T01:41:05.067767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074931+0000","last_change":"2023-10-06T01:41:05.072492+0000","last_active":"2023-10-06T01:41:05.074931+0000","last_peered":"2023-10-06T01:41:05.074931+0000","last_clean":"2023-10-06T01:41:05.074931+0000","last_became_active":"2023-10-06T01:41:05.072225+0000","last_became_peered":"2023-10-06T01:41:05.072225+0000","last_unstale":"2023-10-06T01:41:05.074931+0000","last_undegraded":"2023-10-06T01:41:05.074931+0000","last_fullsized":"2023-10-06T01:41:05.074931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.539127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"36'15","reported_seq":27,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165053+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:40:56.165053+0000","last_peered":"2023-10-06T01:40:56.165053+0000","last_clean":"2023-10-06T01:40:56.165053+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:40:56.165053+0000","last_undegraded":"2023-10-06T01:40:56.165053+0000","last_fullsized":"2023-10-06T01:40:56.165053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164653+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:40:56.164653+0000","last_peered":"2023-10-06T01:40:56.164653+0000","last_clean":"2023-10-06T01:40:56.164653+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:40:56.164653+0000","last_undegraded":"2023-10-06T01:40:56.164653+0000","last_fullsized":"2023-10-06T01:40:56.164653+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067386+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:41:05.067386+0000","last_peered":"2023-10-06T01:41:05.067386+0000","last_clean":"2023-10-06T01:41:05.067386+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:41:05.067386+0000","last_undegraded":"2023-10-06T01:41:05.067386+0000","last_fullsized":"2023-10-06T01:41:05.067386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066926+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:41:05.066926+0000","last_peered":"2023-10-06T01:41:05.066926+0000","last_clean":"2023-10-06T01:41:05.066926+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:41:05.066926+0000","last_undegraded":"2023-10-06T01:41:05.066926+0000","last_fullsized":"2023-10-06T01:41:05.066926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073491+0000","last_change":"2023-10-06T01:41:05.070833+0000","last_active":"2023-10-06T01:41:05.073491+0000","last_peered":"2023-10-06T01:41:05.073491+0000","last_clean":"2023-10-06T01:41:05.073491+0000","last_became_active":"2023-10-06T01:41:05.070486+0000","last_became_peered":"2023-10-06T01:41:05.070486+0000","last_unstale":"2023-10-06T01:41:05.073491+0000","last_undegraded":"2023-10-06T01:41:05.073491+0000","last_fullsized":"2023-10-06T01:41:05.073491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"36'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164917+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:40:56.164917+0000","last_peered":"2023-10-06T01:40:56.164917+0000","last_clean":"2023-10-06T01:40:56.164917+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:40:56.164917+0000","last_undegraded":"2023-10-06T01:40:56.164917+0000","last_fullsized":"2023-10-06T01:40:56.164917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'15","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:41:01.519887+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:41:01.519887+0000","last_peered":"2023-10-06T01:41:01.519887+0000","last_clean":"2023-10-06T01:41:01.519887+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:41:01.519887+0000","last_undegraded":"2023-10-06T01:41:01.519887+0000","last_fullsized":"2023-10-06T01:41:01.519887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071482+0000","last_change":"2023-10-06T01:41:05.069668+0000","last_active":"2023-10-06T01:41:05.071482+0000","last_peered":"2023-10-06T01:41:05.071482+0000","last_clean":"2023-10-06T01:41:05.071482+0000","last_became_active":"2023-10-06T01:41:05.069033+0000","last_became_peered":"2023-10-06T01:41:05.069033+0000","last_unstale":"2023-10-06T01:41:05.071482+0000","last_undegraded":"2023-10-06T01:41:05.071482+0000","last_fullsized":"2023-10-06T01:41:05.071482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:14:32.941476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059936+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:41:05.059936+0000","last_peered":"2023-10-06T01:41:05.059936+0000","last_clean":"2023-10-06T01:41:05.059936+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:41:05.059936+0000","last_undegraded":"2023-10-06T01:41:05.059936+0000","last_fullsized":"2023-10-06T01:41:05.059936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060008+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:41:05.060008+0000","last_peered":"2023-10-06T01:41:05.060008+0000","last_clean":"2023-10-06T01:41:05.060008+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:41:05.060008+0000","last_undegraded":"2023-10-06T01:41:05.060008+0000","last_fullsized":"2023-10-06T01:41:05.060008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075650+0000","last_change":"2023-10-06T01:41:05.072868+0000","last_active":"2023-10-06T01:41:05.075650+0000","last_peered":"2023-10-06T01:41:05.075650+0000","last_clean":"2023-10-06T01:41:05.075650+0000","last_became_active":"2023-10-06T01:41:05.072621+0000","last_became_peered":"2023-10-06T01:41:05.072621+0000","last_unstale":"2023-10-06T01:41:05.075650+0000","last_undegraded":"2023-10-06T01:41:05.075650+0000","last_fullsized":"2023-10-06T01:41:05.075650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165859+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:40:56.165859+0000","last_peered":"2023-10-06T01:40:56.165859+0000","last_clean":"2023-10-06T01:40:56.165859+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:40:56.165859+0000","last_undegraded":"2023-10-06T01:40:56.165859+0000","last_fullsized":"2023-10-06T01:40:56.165859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'3","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063389+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:41:05.063389+0000","last_peered":"2023-10-06T01:41:05.063389+0000","last_clean":"2023-10-06T01:41:05.063389+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:41:05.063389+0000","last_undegraded":"2023-10-06T01:41:05.063389+0000","last_fullsized":"2023-10-06T01:41:05.063389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.080676+0000","last_change":"2023-10-06T01:41:05.073304+0000","last_active":"2023-10-06T01:41:05.080676+0000","last_peered":"2023-10-06T01:41:05.080676+0000","last_clean":"2023-10-06T01:41:05.080676+0000","last_became_active":"2023-10-06T01:41:05.073056+0000","last_became_peered":"2023-10-06T01:41:05.073056+0000","last_unstale":"2023-10-06T01:41:05.080676+0000","last_undegraded":"2023-10-06T01:41:05.080676+0000","last_fullsized":"2023-10-06T01:41:05.080676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:14.536410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"36'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041175+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:40:56.041175+0000","last_peered":"2023-10-06T01:40:56.041175+0000","last_clean":"2023-10-06T01:40:56.041175+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:40:56.041175+0000","last_undegraded":"2023-10-06T01:40:56.041175+0000","last_fullsized":"2023-10-06T01:40:56.041175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'2","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067860+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:41:05.067860+0000","last_peered":"2023-10-06T01:41:05.067860+0000","last_clean":"2023-10-06T01:41:05.067860+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:41:05.067860+0000","last_undegraded":"2023-10-06T01:41:05.067860+0000","last_fullsized":"2023-10-06T01:41:05.067860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068813+0000","last_change":"2023-10-06T01:41:05.066269+0000","last_active":"2023-10-06T01:41:05.068813+0000","last_peered":"2023-10-06T01:41:05.068813+0000","last_clean":"2023-10-06T01:41:05.068813+0000","last_became_active":"2023-10-06T01:41:05.065979+0000","last_became_peered":"2023-10-06T01:41:05.065979+0000","last_unstale":"2023-10-06T01:41:05.068813+0000","last_undegraded":"2023-10-06T01:41:05.068813+0000","last_fullsized":"2023-10-06T01:41:05.068813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"36'7","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041494+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:40:56.041494+0000","last_peered":"2023-10-06T01:40:56.041494+0000","last_clean":"2023-10-06T01:40:56.041494+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:40:56.041494+0000","last_undegraded":"2023-10-06T01:40:56.041494+0000","last_fullsized":"2023-10-06T01:40:56.041494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067544+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:41:05.067544+0000","last_peered":"2023-10-06T01:41:05.067544+0000","last_clean":"2023-10-06T01:41:05.067544+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:41:05.067544+0000","last_undegraded":"2023-10-06T01:41:05.067544+0000","last_fullsized":"2023-10-06T01:41:05.067544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067040+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:41:05.067040+0000","last_peered":"2023-10-06T01:41:05.067040+0000","last_clean":"2023-10-06T01:41:05.067040+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:41:05.067040+0000","last_undegraded":"2023-10-06T01:41:05.067040+0000","last_fullsized":"2023-10-06T01:41:05.067040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'4","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067848+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:41:05.067848+0000","last_peered":"2023-10-06T01:41:05.067848+0000","last_clean":"2023-10-06T01:41:05.067848+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:41:05.067848+0000","last_undegraded":"2023-10-06T01:41:05.067848+0000","last_fullsized":"2023-10-06T01:41:05.067848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074693+0000","last_change":"2023-10-06T01:41:05.069995+0000","last_active":"2023-10-06T01:41:05.074693+0000","last_peered":"2023-10-06T01:41:05.074693+0000","last_clean":"2023-10-06T01:41:05.074693+0000","last_became_active":"2023-10-06T01:41:05.069709+0000","last_became_peered":"2023-10-06T01:41:05.069709+0000","last_unstale":"2023-10-06T01:41:05.074693+0000","last_undegraded":"2023-10-06T01:41:05.074693+0000","last_fullsized":"2023-10-06T01:41:05.074693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:50.555907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062679+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:41:05.062679+0000","last_peered":"2023-10-06T01:41:05.062679+0000","last_clean":"2023-10-06T01:41:05.062679+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:41:05.062679+0000","last_undegraded":"2023-10-06T01:41:05.062679+0000","last_fullsized":"2023-10-06T01:41:05.062679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066603+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:41:05.066603+0000","last_peered":"2023-10-06T01:41:05.066603+0000","last_clean":"2023-10-06T01:41:05.066603+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:41:05.066603+0000","last_undegraded":"2023-10-06T01:41:05.066603+0000","last_fullsized":"2023-10-06T01:41:05.066603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077363+0000","last_change":"2023-10-06T01:41:05.071858+0000","last_active":"2023-10-06T01:41:05.077363+0000","last_peered":"2023-10-06T01:41:05.077363+0000","last_clean":"2023-10-06T01:41:05.077363+0000","last_became_active":"2023-10-06T01:41:05.071590+0000","last_became_peered":"2023-10-06T01:41:05.071590+0000","last_unstale":"2023-10-06T01:41:05.077363+0000","last_undegraded":"2023-10-06T01:41:05.077363+0000","last_fullsized":"2023-10-06T01:41:05.077363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:39.551110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059831+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:41:05.059831+0000","last_peered":"2023-10-06T01:41:05.059831+0000","last_clean":"2023-10-06T01:41:05.059831+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:41:05.059831+0000","last_undegraded":"2023-10-06T01:41:05.059831+0000","last_fullsized":"2023-10-06T01:41:05.059831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164866+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:40:56.164866+0000","last_peered":"2023-10-06T01:40:56.164866+0000","last_clean":"2023-10-06T01:40:56.164866+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:40:56.164866+0000","last_undegraded":"2023-10-06T01:40:56.164866+0000","last_fullsized":"2023-10-06T01:40:56.164866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"36'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164922+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:40:56.164922+0000","last_peered":"2023-10-06T01:40:56.164922+0000","last_clean":"2023-10-06T01:40:56.164922+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:40:56.164922+0000","last_undegraded":"2023-10-06T01:40:56.164922+0000","last_fullsized":"2023-10-06T01:40:56.164922+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164422+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:40:56.164422+0000","last_peered":"2023-10-06T01:40:56.164422+0000","last_clean":"2023-10-06T01:40:56.164422+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:40:56.164422+0000","last_undegraded":"2023-10-06T01:40:56.164422+0000","last_fullsized":"2023-10-06T01:40:56.164422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067681+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:41:05.067681+0000","last_peered":"2023-10-06T01:41:05.067681+0000","last_clean":"2023-10-06T01:41:05.067681+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:41:05.067681+0000","last_undegraded":"2023-10-06T01:41:05.067681+0000","last_fullsized":"2023-10-06T01:41:05.067681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063996+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:41:05.063996+0000","last_peered":"2023-10-06T01:41:05.063996+0000","last_clean":"2023-10-06T01:41:05.063996+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:41:05.063996+0000","last_undegraded":"2023-10-06T01:41:05.063996+0000","last_fullsized":"2023-10-06T01:41:05.063996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062842+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:05.062842+0000","last_peered":"2023-10-06T01:41:05.062842+0000","last_clean":"2023-10-06T01:41:05.062842+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:05.062842+0000","last_undegraded":"2023-10-06T01:41:05.062842+0000","last_fullsized":"2023-10-06T01:41:05.062842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074573+0000","last_change":"2023-10-06T01:41:05.070827+0000","last_active":"2023-10-06T01:41:05.074573+0000","last_peered":"2023-10-06T01:41:05.074573+0000","last_clean":"2023-10-06T01:41:05.074573+0000","last_became_active":"2023-10-06T01:41:05.070649+0000","last_became_peered":"2023-10-06T01:41:05.070649+0000","last_unstale":"2023-10-06T01:41:05.074573+0000","last_undegraded":"2023-10-06T01:41:05.074573+0000","last_fullsized":"2023-10-06T01:41:05.074573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:26.468333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067196+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:41:05.067196+0000","last_peered":"2023-10-06T01:41:05.067196+0000","last_clean":"2023-10-06T01:41:05.067196+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:41:05.067196+0000","last_undegraded":"2023-10-06T01:41:05.067196+0000","last_fullsized":"2023-10-06T01:41:05.067196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165177+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:40:56.165177+0000","last_peered":"2023-10-06T01:40:56.165177+0000","last_clean":"2023-10-06T01:40:56.165177+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:40:56.165177+0000","last_undegraded":"2023-10-06T01:40:56.165177+0000","last_fullsized":"2023-10-06T01:40:56.165177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067611+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:05.067611+0000","last_peered":"2023-10-06T01:41:05.067611+0000","last_clean":"2023-10-06T01:41:05.067611+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:05.067611+0000","last_undegraded":"2023-10-06T01:41:05.067611+0000","last_fullsized":"2023-10-06T01:41:05.067611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071384+0000","last_change":"2023-10-06T01:41:05.068635+0000","last_active":"2023-10-06T01:41:05.071384+0000","last_peered":"2023-10-06T01:41:05.071384+0000","last_clean":"2023-10-06T01:41:05.071384+0000","last_became_active":"2023-10-06T01:41:05.068364+0000","last_became_peered":"2023-10-06T01:41:05.068364+0000","last_unstale":"2023-10-06T01:41:05.071384+0000","last_undegraded":"2023-10-06T01:41:05.071384+0000","last_fullsized":"2023-10-06T01:41:05.071384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:53.027266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"36'5","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164843+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:40:56.164843+0000","last_peered":"2023-10-06T01:40:56.164843+0000","last_clean":"2023-10-06T01:40:56.164843+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:40:56.164843+0000","last_undegraded":"2023-10-06T01:40:56.164843+0000","last_fullsized":"2023-10-06T01:40:56.164843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062581+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:41:05.062581+0000","last_peered":"2023-10-06T01:41:05.062581+0000","last_clean":"2023-10-06T01:41:05.062581+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:41:05.062581+0000","last_undegraded":"2023-10-06T01:41:05.062581+0000","last_fullsized":"2023-10-06T01:41:05.062581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":178,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164776+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:40:56.164776+0000","last_peered":"2023-10-06T01:40:56.164776+0000","last_clean":"2023-10-06T01:40:56.164776+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:40:56.164776+0000","last_undegraded":"2023-10-06T01:40:56.164776+0000","last_fullsized":"2023-10-06T01:40:56.164776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075630+0000","last_change":"2023-10-06T01:41:05.072823+0000","last_active":"2023-10-06T01:41:05.075630+0000","last_peered":"2023-10-06T01:41:05.075630+0000","last_clean":"2023-10-06T01:41:05.075630+0000","last_became_active":"2023-10-06T01:41:05.072511+0000","last_became_peered":"2023-10-06T01:41:05.072511+0000","last_unstale":"2023-10-06T01:41:05.075630+0000","last_undegraded":"2023-10-06T01:41:05.075630+0000","last_fullsized":"2023-10-06T01:41:05.075630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"36'9","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165361+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:40:56.165361+0000","last_peered":"2023-10-06T01:40:56.165361+0000","last_clean":"2023-10-06T01:40:56.165361+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:40:56.165361+0000","last_undegraded":"2023-10-06T01:40:56.165361+0000","last_fullsized":"2023-10-06T01:40:56.165361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164979+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:40:56.164979+0000","last_peered":"2023-10-06T01:40:56.164979+0000","last_clean":"2023-10-06T01:40:56.164979+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:40:56.164979+0000","last_undegraded":"2023-10-06T01:40:56.164979+0000","last_fullsized":"2023-10-06T01:40:56.164979+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164838+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:40:56.164838+0000","last_peered":"2023-10-06T01:40:56.164838+0000","last_clean":"2023-10-06T01:40:56.164838+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:40:56.164838+0000","last_undegraded":"2023-10-06T01:40:56.164838+0000","last_fullsized":"2023-10-06T01:40:56.164838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"36'7","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165559+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:40:56.165559+0000","last_peered":"2023-10-06T01:40:56.165559+0000","last_clean":"2023-10-06T01:40:56.165559+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:40:56.165559+0000","last_undegraded":"2023-10-06T01:40:56.165559+0000","last_fullsized":"2023-10-06T01:40:56.165559+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068333+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:41:05.068333+0000","last_peered":"2023-10-06T01:41:05.068333+0000","last_clean":"2023-10-06T01:41:05.068333+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:41:05.068333+0000","last_undegraded":"2023-10-06T01:41:05.068333+0000","last_fullsized":"2023-10-06T01:41:05.068333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.164836+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:40:56.164836+0000","last_peered":"2023-10-06T01:40:56.164836+0000","last_clean":"2023-10-06T01:40:56.164836+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:40:56.164836+0000","last_undegraded":"2023-10-06T01:40:56.164836+0000","last_fullsized":"2023-10-06T01:40:56.164836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074585+0000","last_change":"2023-10-06T01:41:05.071521+0000","last_active":"2023-10-06T01:41:05.074585+0000","last_peered":"2023-10-06T01:41:05.074585+0000","last_clean":"2023-10-06T01:41:05.074585+0000","last_became_active":"2023-10-06T01:41:05.071272+0000","last_became_peered":"2023-10-06T01:41:05.071272+0000","last_unstale":"2023-10-06T01:41:05.074585+0000","last_undegraded":"2023-10-06T01:41:05.074585+0000","last_fullsized":"2023-10-06T01:41:05.074585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.431329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"36'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041758+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:40:56.041758+0000","last_peered":"2023-10-06T01:40:56.041758+0000","last_clean":"2023-10-06T01:40:56.041758+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:40:56.041758+0000","last_undegraded":"2023-10-06T01:40:56.041758+0000","last_fullsized":"2023-10-06T01:40:56.041758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'3","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068093+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:41:05.068093+0000","last_peered":"2023-10-06T01:41:05.068093+0000","last_clean":"2023-10-06T01:41:05.068093+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:41:05.068093+0000","last_undegraded":"2023-10-06T01:41:05.068093+0000","last_fullsized":"2023-10-06T01:41:05.068093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066856+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:05.066856+0000","last_peered":"2023-10-06T01:41:05.066856+0000","last_clean":"2023-10-06T01:41:05.066856+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:05.066856+0000","last_undegraded":"2023-10-06T01:41:05.066856+0000","last_fullsized":"2023-10-06T01:41:05.066856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077722+0000","last_change":"2023-10-06T01:41:05.073694+0000","last_active":"2023-10-06T01:41:05.077722+0000","last_peered":"2023-10-06T01:41:05.077722+0000","last_clean":"2023-10-06T01:41:05.077722+0000","last_became_active":"2023-10-06T01:41:05.073261+0000","last_became_peered":"2023-10-06T01:41:05.073261+0000","last_unstale":"2023-10-06T01:41:05.077722+0000","last_undegraded":"2023-10-06T01:41:05.077722+0000","last_fullsized":"2023-10-06T01:41:05.077722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066331+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:41:05.066331+0000","last_peered":"2023-10-06T01:41:05.066331+0000","last_clean":"2023-10-06T01:41:05.066331+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:41:05.066331+0000","last_undegraded":"2023-10-06T01:41:05.066331+0000","last_fullsized":"2023-10-06T01:41:05.066331+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062956+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:41:05.062956+0000","last_peered":"2023-10-06T01:41:05.062956+0000","last_clean":"2023-10-06T01:41:05.062956+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:41:05.062956+0000","last_undegraded":"2023-10-06T01:41:05.062956+0000","last_fullsized":"2023-10-06T01:41:05.062956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041323+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:40:56.041323+0000","last_peered":"2023-10-06T01:40:56.041323+0000","last_clean":"2023-10-06T01:40:56.041323+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:40:56.041323+0000","last_undegraded":"2023-10-06T01:40:56.041323+0000","last_fullsized":"2023-10-06T01:40:56.041323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T01:41:04.051667+0000","last_change":"2023-10-06T01:41:04.051667+0000","last_active":"2023-10-06T01:41:04.051667+0000","last_peered":"2023-10-06T01:41:04.051667+0000","last_clean":"2023-10-06T01:41:04.051667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T01:41:04.051667+0000","last_undegraded":"2023-10-06T01:41:04.051667+0000","last_fullsized":"2023-10-06T01:41:04.051667+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"36'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062666+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:41:05.062666+0000","last_peered":"2023-10-06T01:41:05.062666+0000","last_clean":"2023-10-06T01:41:05.062666+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:41:05.062666+0000","last_undegraded":"2023-10-06T01:41:05.062666+0000","last_fullsized":"2023-10-06T01:41:05.062666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.165539+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:40:56.165539+0000","last_peered":"2023-10-06T01:40:56.165539+0000","last_clean":"2023-10-06T01:40:56.165539+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:40:56.165539+0000","last_undegraded":"2023-10-06T01:40:56.165539+0000","last_fullsized":"2023-10-06T01:40:56.165539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T01:40:56.041835+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:40:56.041835+0000","last_peered":"2023-10-06T01:40:56.041835+0000","last_clean":"2023-10-06T01:40:56.041835+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:40:56.041835+0000","last_undegraded":"2023-10-06T01:40:56.041835+0000","last_fullsized":"2023-10-06T01:40:56.041835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":40,"acting":40,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298332,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073508,"statfs":{"total":96636764160,"available":96331272192,"internally_reserved":0,"allocated":1069056,"data_stored":806541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.88},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4669440,"data_stored":4410477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.498}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.309}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4669440,"data_stored":4410477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51300000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":237748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.312}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":239101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41199999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.622},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:09.492 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:09.637 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.636+0000 7f771ec4d700 1 Processor -- start 2023-10-06T01:41:09.638 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.636+0000 7f771ec4d700 1 -- start start 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771ec4d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7718159480 0x7f77180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771ec4d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f77180b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771ec4d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77180cb7e0 con 0x7f77180b3610 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771ec4d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180b08a0 con 0x7f7718159480 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771ec4d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180cb190 con 0x7f77180b2140 2023-10-06T01:41:09.640 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7718159480 0x7f77180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7718159480 0x7f77180b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55150/0 (socket says 172.21.15.98:55150) 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 -- 172.21.15.98:0/549824863 learned_addr learned my addr 172.21.15.98:0/549824863 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771e44c700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f77180b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 -- 172.21.15.98:0/549824863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 msgr2=0x7f77180b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f77180b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 -- 172.21.15.98:0/549824863 >> v1:172.21.15.184:6789/0 conn(0x7f77180b3610 legacy=0x7f77180cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.637+0000 7f771dc4b700 1 -- 172.21.15.98:0/549824863 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180cb470 con 0x7f7718159480 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771dc4b700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f77180b30d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9607afdf4df8e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771cc49700 1 -- 172.21.15.98:0/549824863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714009ea0 con 0x7f7718159480 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771cc49700 1 -- 172.21.15.98:0/549824863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771400b860 con 0x7f7718159480 2023-10-06T01:41:09.641 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771cc49700 1 -- 172.21.15.98:0/549824863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400bc00 con 0x7f7718159480 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 -- 172.21.15.98:0/549824863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 msgr2=0x7f77180b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f77180b30d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714009250 comp rx=0 tx=0).stop 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 -- 172.21.15.98:0/549824863 shutdown_connections 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f77180b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 --2- 172.21.15.98:0/549824863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f77180b30d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 -- 172.21.15.98:0/549824863 >> 172.21.15.98:0/549824863 conn(0x7f771806ed20 msgr2=0x7f771806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 -- 172.21.15.98:0/549824863 shutdown_connections 2023-10-06T01:41:09.642 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.638+0000 7f771ec4d700 1 -- 172.21.15.98:0/549824863 wait complete. 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.639+0000 7f771ec4d700 1 Processor -- start 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.639+0000 7f771ec4d700 1 -- start start 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f7718154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77180b3610 0x7f771815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718159b80 con 0x7f7718159480 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c8650 con 0x7f77180b3610 2023-10-06T01:41:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771ec4d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771815bb60 con 0x7f77180b2140 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771e44c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771e44c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55156/0 (socket says 172.21.15.98:55156) 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771e44c700 1 -- 172.21.15.98:0/481730130 learned_addr learned my addr 172.21.15.98:0/481730130 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771d44a700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77180b3610 0x7f771815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.640+0000 7f771dc4b700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f7718154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 -- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 msgr2=0x7f7718154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f7718154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 -- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77180b3610 msgr2=0x7f771815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77180b3610 0x7f771815af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 -- 172.21.15.98:0/481730130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140094f0 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771e44c700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e90 tx=0x7f771000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000cb70 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000f040 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710014480 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771815be40 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.641+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7718169ca0 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.642+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77180c8490 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.642+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7710014830 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.643+0000 7f770dffb700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 0x7f76fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.643+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f7710085420 con 0x7f7718159480 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.643+0000 7f771dc4b700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 0x7f76fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.643+0000 7f771dc4b700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 0x7f76fc0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:09.648 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.647+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77180c8490 con 0x7f7718159480 2023-10-06T01:41:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.826+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77180c8490 con 0x7f76fc066f30 2023-10-06T01:41:09.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.836+0000 7f770dffb700 1 -- 172.21.15.98:0/481730130 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+392321 (secure 0 0 0) 0x7f77180c8490 con 0x7f76fc066f30 2023-10-06T01:41:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 msgr2=0x7f76fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 0x7f76fc0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714017000 comp rx=0 tx=0).stop 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 msgr2=0x7f771815b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e90 tx=0x7f771000b040 comp rx=0 tx=0).stop 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 shutdown_connections 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f76fc066f30 0x7f76fc0693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7718159480 0x7f771815b4c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77180b2140 0x7f7718154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 --2- 172.21.15.98:0/481730130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77180b3610 0x7f771815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 >> 172.21.15.98:0/481730130 conn(0x7f771806ed20 msgr2=0x7f77180cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 shutdown_connections 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:09.840+0000 7f771ec4d700 1 -- 172.21.15.98:0/481730130 wait complete. 2023-10-06T01:41:09.846 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:09.853 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-06T01:41:09.802971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":255,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395748,"kb_used_data":17444,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578972,"statfs":{"total":773094113280,"available":770640867328,"internally_reserved":0,"allocated":17862656,"data_stored":15675103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001235"},"pg_stats":[{"pgid":"7.3f","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070953+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:41:05.070953+0000","last_peered":"2023-10-06T01:41:05.070953+0000","last_clean":"2023-10-06T01:41:05.070953+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:41:05.070953+0000","last_undegraded":"2023-10-06T01:41:05.070953+0000","last_fullsized":"2023-10-06T01:41:05.070953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068055+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:41:05.068055+0000","last_peered":"2023-10-06T01:41:05.068055+0000","last_clean":"2023-10-06T01:41:05.068055+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:41:05.068055+0000","last_undegraded":"2023-10-06T01:41:05.068055+0000","last_fullsized":"2023-10-06T01:41:05.068055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062731+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:41:05.062731+0000","last_peered":"2023-10-06T01:41:05.062731+0000","last_clean":"2023-10-06T01:41:05.062731+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:41:05.062731+0000","last_undegraded":"2023-10-06T01:41:05.062731+0000","last_fullsized":"2023-10-06T01:41:05.062731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064200+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:41:05.064200+0000","last_peered":"2023-10-06T01:41:05.064200+0000","last_clean":"2023-10-06T01:41:05.064200+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:41:05.064200+0000","last_undegraded":"2023-10-06T01:41:05.064200+0000","last_fullsized":"2023-10-06T01:41:05.064200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067188+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:41:05.067188+0000","last_peered":"2023-10-06T01:41:05.067188+0000","last_clean":"2023-10-06T01:41:05.067188+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:41:05.067188+0000","last_undegraded":"2023-10-06T01:41:05.067188+0000","last_fullsized":"2023-10-06T01:41:05.067188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067168+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:41:05.067168+0000","last_peered":"2023-10-06T01:41:05.067168+0000","last_clean":"2023-10-06T01:41:05.067168+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:41:05.067168+0000","last_undegraded":"2023-10-06T01:41:05.067168+0000","last_fullsized":"2023-10-06T01:41:05.067168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068070+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:41:05.068070+0000","last_peered":"2023-10-06T01:41:05.068070+0000","last_clean":"2023-10-06T01:41:05.068070+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:41:05.068070+0000","last_undegraded":"2023-10-06T01:41:05.068070+0000","last_fullsized":"2023-10-06T01:41:05.068070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063265+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:41:05.063265+0000","last_peered":"2023-10-06T01:41:05.063265+0000","last_clean":"2023-10-06T01:41:05.063265+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:41:05.063265+0000","last_undegraded":"2023-10-06T01:41:05.063265+0000","last_fullsized":"2023-10-06T01:41:05.063265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"36'11","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067676+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:41:05.067676+0000","last_peered":"2023-10-06T01:41:05.067676+0000","last_clean":"2023-10-06T01:41:05.067676+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:41:05.067676+0000","last_undegraded":"2023-10-06T01:41:05.067676+0000","last_fullsized":"2023-10-06T01:41:05.067676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064867+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:41:05.064867+0000","last_peered":"2023-10-06T01:41:05.064867+0000","last_clean":"2023-10-06T01:41:05.064867+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:41:05.064867+0000","last_undegraded":"2023-10-06T01:41:05.064867+0000","last_fullsized":"2023-10-06T01:41:05.064867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064219+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:41:05.064219+0000","last_peered":"2023-10-06T01:41:05.064219+0000","last_clean":"2023-10-06T01:41:05.064219+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:41:05.064219+0000","last_undegraded":"2023-10-06T01:41:05.064219+0000","last_fullsized":"2023-10-06T01:41:05.064219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066867+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:41:05.066867+0000","last_peered":"2023-10-06T01:41:05.066867+0000","last_clean":"2023-10-06T01:41:05.066867+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:41:05.066867+0000","last_undegraded":"2023-10-06T01:41:05.066867+0000","last_fullsized":"2023-10-06T01:41:05.066867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"36'10","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059981+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:41:05.059981+0000","last_peered":"2023-10-06T01:41:05.059981+0000","last_clean":"2023-10-06T01:41:05.059981+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:41:05.059981+0000","last_undegraded":"2023-10-06T01:41:05.059981+0000","last_fullsized":"2023-10-06T01:41:05.059981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063679+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:41:05.063679+0000","last_peered":"2023-10-06T01:41:05.063679+0000","last_clean":"2023-10-06T01:41:05.063679+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:41:05.063679+0000","last_undegraded":"2023-10-06T01:41:05.063679+0000","last_fullsized":"2023-10-06T01:41:05.063679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066663+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:41:05.066663+0000","last_peered":"2023-10-06T01:41:05.066663+0000","last_clean":"2023-10-06T01:41:05.066663+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:41:05.066663+0000","last_undegraded":"2023-10-06T01:41:05.066663+0000","last_fullsized":"2023-10-06T01:41:05.066663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061713+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:41:05.061713+0000","last_peered":"2023-10-06T01:41:05.061713+0000","last_clean":"2023-10-06T01:41:05.061713+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:41:05.061713+0000","last_undegraded":"2023-10-06T01:41:05.061713+0000","last_fullsized":"2023-10-06T01:41:05.061713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063591+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:41:05.063591+0000","last_peered":"2023-10-06T01:41:05.063591+0000","last_clean":"2023-10-06T01:41:05.063591+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:41:05.063591+0000","last_undegraded":"2023-10-06T01:41:05.063591+0000","last_fullsized":"2023-10-06T01:41:05.063591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062553+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:41:05.062553+0000","last_peered":"2023-10-06T01:41:05.062553+0000","last_clean":"2023-10-06T01:41:05.062553+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:41:05.062553+0000","last_undegraded":"2023-10-06T01:41:05.062553+0000","last_fullsized":"2023-10-06T01:41:05.062553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062556+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:41:05.062556+0000","last_peered":"2023-10-06T01:41:05.062556+0000","last_clean":"2023-10-06T01:41:05.062556+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:41:05.062556+0000","last_undegraded":"2023-10-06T01:41:05.062556+0000","last_fullsized":"2023-10-06T01:41:05.062556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059961+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:41:05.059961+0000","last_peered":"2023-10-06T01:41:05.059961+0000","last_clean":"2023-10-06T01:41:05.059961+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:41:05.059961+0000","last_undegraded":"2023-10-06T01:41:05.059961+0000","last_fullsized":"2023-10-06T01:41:05.059961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067584+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:41:05.067584+0000","last_peered":"2023-10-06T01:41:05.067584+0000","last_clean":"2023-10-06T01:41:05.067584+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:41:05.067584+0000","last_undegraded":"2023-10-06T01:41:05.067584+0000","last_fullsized":"2023-10-06T01:41:05.067584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063813+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:41:05.063813+0000","last_peered":"2023-10-06T01:41:05.063813+0000","last_clean":"2023-10-06T01:41:05.063813+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:41:05.063813+0000","last_undegraded":"2023-10-06T01:41:05.063813+0000","last_fullsized":"2023-10-06T01:41:05.063813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"36'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062072+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:41:05.062072+0000","last_peered":"2023-10-06T01:41:05.062072+0000","last_clean":"2023-10-06T01:41:05.062072+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:41:05.062072+0000","last_undegraded":"2023-10-06T01:41:05.062072+0000","last_fullsized":"2023-10-06T01:41:05.062072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"36'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059914+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:41:05.059914+0000","last_peered":"2023-10-06T01:41:05.059914+0000","last_clean":"2023-10-06T01:41:05.059914+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:41:05.059914+0000","last_undegraded":"2023-10-06T01:41:05.059914+0000","last_fullsized":"2023-10-06T01:41:05.059914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062784+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:41:05.062784+0000","last_peered":"2023-10-06T01:41:05.062784+0000","last_clean":"2023-10-06T01:41:05.062784+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:41:05.062784+0000","last_undegraded":"2023-10-06T01:41:05.062784+0000","last_fullsized":"2023-10-06T01:41:05.062784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067606+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:41:05.067606+0000","last_peered":"2023-10-06T01:41:05.067606+0000","last_clean":"2023-10-06T01:41:05.067606+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:41:05.067606+0000","last_undegraded":"2023-10-06T01:41:05.067606+0000","last_fullsized":"2023-10-06T01:41:05.067606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061786+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:41:05.061786+0000","last_peered":"2023-10-06T01:41:05.061786+0000","last_clean":"2023-10-06T01:41:05.061786+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:41:05.061786+0000","last_undegraded":"2023-10-06T01:41:05.061786+0000","last_fullsized":"2023-10-06T01:41:05.061786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063049+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:41:05.063049+0000","last_peered":"2023-10-06T01:41:05.063049+0000","last_clean":"2023-10-06T01:41:05.063049+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:41:05.063049+0000","last_undegraded":"2023-10-06T01:41:05.063049+0000","last_fullsized":"2023-10-06T01:41:05.063049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"36'12","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066771+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:41:05.066771+0000","last_peered":"2023-10-06T01:41:05.066771+0000","last_clean":"2023-10-06T01:41:05.066771+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:41:05.066771+0000","last_undegraded":"2023-10-06T01:41:05.066771+0000","last_fullsized":"2023-10-06T01:41:05.066771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059829+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:41:05.059829+0000","last_peered":"2023-10-06T01:41:05.059829+0000","last_clean":"2023-10-06T01:41:05.059829+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:41:05.059829+0000","last_undegraded":"2023-10-06T01:41:05.059829+0000","last_fullsized":"2023-10-06T01:41:05.059829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074919+0000","last_change":"2023-10-06T01:41:05.072432+0000","last_active":"2023-10-06T01:41:05.074919+0000","last_peered":"2023-10-06T01:41:05.074919+0000","last_clean":"2023-10-06T01:41:05.074919+0000","last_became_active":"2023-10-06T01:41:05.072154+0000","last_became_peered":"2023-10-06T01:41:05.072154+0000","last_unstale":"2023-10-06T01:41:05.074919+0000","last_undegraded":"2023-10-06T01:41:05.074919+0000","last_fullsized":"2023-10-06T01:41:05.074919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:55.047887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063306+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:41:05.063306+0000","last_peered":"2023-10-06T01:41:05.063306+0000","last_clean":"2023-10-06T01:41:05.063306+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:41:05.063306+0000","last_undegraded":"2023-10-06T01:41:05.063306+0000","last_fullsized":"2023-10-06T01:41:05.063306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068615+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:41:05.068615+0000","last_peered":"2023-10-06T01:41:05.068615+0000","last_clean":"2023-10-06T01:41:05.068615+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:41:05.068615+0000","last_undegraded":"2023-10-06T01:41:05.068615+0000","last_fullsized":"2023-10-06T01:41:05.068615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077358+0000","last_change":"2023-10-06T01:41:05.075392+0000","last_active":"2023-10-06T01:41:05.077358+0000","last_peered":"2023-10-06T01:41:05.077358+0000","last_clean":"2023-10-06T01:41:05.077358+0000","last_became_active":"2023-10-06T01:41:05.075108+0000","last_became_peered":"2023-10-06T01:41:05.075108+0000","last_unstale":"2023-10-06T01:41:05.077358+0000","last_undegraded":"2023-10-06T01:41:05.077358+0000","last_fullsized":"2023-10-06T01:41:05.077358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:43.179045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061850+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:41:05.061850+0000","last_peered":"2023-10-06T01:41:05.061850+0000","last_clean":"2023-10-06T01:41:05.061850+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:41:05.061850+0000","last_undegraded":"2023-10-06T01:41:05.061850+0000","last_fullsized":"2023-10-06T01:41:05.061850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063096+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:41:05.063096+0000","last_peered":"2023-10-06T01:41:05.063096+0000","last_clean":"2023-10-06T01:41:05.063096+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:41:05.063096+0000","last_undegraded":"2023-10-06T01:41:05.063096+0000","last_fullsized":"2023-10-06T01:41:05.063096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075586+0000","last_change":"2023-10-06T01:41:05.072321+0000","last_active":"2023-10-06T01:41:05.075586+0000","last_peered":"2023-10-06T01:41:05.075586+0000","last_clean":"2023-10-06T01:41:05.075586+0000","last_became_active":"2023-10-06T01:41:05.072037+0000","last_became_peered":"2023-10-06T01:41:05.072037+0000","last_unstale":"2023-10-06T01:41:05.075586+0000","last_undegraded":"2023-10-06T01:41:05.075586+0000","last_fullsized":"2023-10-06T01:41:05.075586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067494+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:41:05.067494+0000","last_peered":"2023-10-06T01:41:05.067494+0000","last_clean":"2023-10-06T01:41:05.067494+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:41:05.067494+0000","last_undegraded":"2023-10-06T01:41:05.067494+0000","last_fullsized":"2023-10-06T01:41:05.067494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062861+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:41:05.062861+0000","last_peered":"2023-10-06T01:41:05.062861+0000","last_clean":"2023-10-06T01:41:05.062861+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:41:05.062861+0000","last_undegraded":"2023-10-06T01:41:05.062861+0000","last_fullsized":"2023-10-06T01:41:05.062861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071654+0000","last_change":"2023-10-06T01:41:05.070017+0000","last_active":"2023-10-06T01:41:05.071654+0000","last_peered":"2023-10-06T01:41:05.071654+0000","last_clean":"2023-10-06T01:41:05.071654+0000","last_became_active":"2023-10-06T01:41:05.069741+0000","last_became_peered":"2023-10-06T01:41:05.069741+0000","last_unstale":"2023-10-06T01:41:05.071654+0000","last_undegraded":"2023-10-06T01:41:05.071654+0000","last_fullsized":"2023-10-06T01:41:05.071654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:36.570717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062619+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:41:05.062619+0000","last_peered":"2023-10-06T01:41:05.062619+0000","last_clean":"2023-10-06T01:41:05.062619+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:41:05.062619+0000","last_undegraded":"2023-10-06T01:41:05.062619+0000","last_fullsized":"2023-10-06T01:41:05.062619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067659+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:41:05.067659+0000","last_peered":"2023-10-06T01:41:05.067659+0000","last_clean":"2023-10-06T01:41:05.067659+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:41:05.067659+0000","last_undegraded":"2023-10-06T01:41:05.067659+0000","last_fullsized":"2023-10-06T01:41:05.067659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077336+0000","last_change":"2023-10-06T01:41:05.074701+0000","last_active":"2023-10-06T01:41:05.077336+0000","last_peered":"2023-10-06T01:41:05.077336+0000","last_clean":"2023-10-06T01:41:05.077336+0000","last_became_active":"2023-10-06T01:41:05.074479+0000","last_became_peered":"2023-10-06T01:41:05.074479+0000","last_unstale":"2023-10-06T01:41:05.077336+0000","last_undegraded":"2023-10-06T01:41:05.077336+0000","last_fullsized":"2023-10-06T01:41:05.077336+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:42.689091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063152+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:41:05.063152+0000","last_peered":"2023-10-06T01:41:05.063152+0000","last_clean":"2023-10-06T01:41:05.063152+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:41:05.063152+0000","last_undegraded":"2023-10-06T01:41:05.063152+0000","last_fullsized":"2023-10-06T01:41:05.063152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067927+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:41:05.067927+0000","last_peered":"2023-10-06T01:41:05.067927+0000","last_clean":"2023-10-06T01:41:05.067927+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:41:05.067927+0000","last_undegraded":"2023-10-06T01:41:05.067927+0000","last_fullsized":"2023-10-06T01:41:05.067927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071094+0000","last_change":"2023-10-06T01:41:05.068576+0000","last_active":"2023-10-06T01:41:05.071094+0000","last_peered":"2023-10-06T01:41:05.071094+0000","last_clean":"2023-10-06T01:41:05.071094+0000","last_became_active":"2023-10-06T01:41:05.068329+0000","last_became_peered":"2023-10-06T01:41:05.068329+0000","last_unstale":"2023-10-06T01:41:05.071094+0000","last_undegraded":"2023-10-06T01:41:05.071094+0000","last_fullsized":"2023-10-06T01:41:05.071094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063824+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:41:05.063824+0000","last_peered":"2023-10-06T01:41:05.063824+0000","last_clean":"2023-10-06T01:41:05.063824+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:41:05.063824+0000","last_undegraded":"2023-10-06T01:41:05.063824+0000","last_fullsized":"2023-10-06T01:41:05.063824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063903+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:41:05.063903+0000","last_peered":"2023-10-06T01:41:05.063903+0000","last_clean":"2023-10-06T01:41:05.063903+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:41:05.063903+0000","last_undegraded":"2023-10-06T01:41:05.063903+0000","last_fullsized":"2023-10-06T01:41:05.063903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.076999+0000","last_change":"2023-10-06T01:41:05.072828+0000","last_active":"2023-10-06T01:41:05.076999+0000","last_peered":"2023-10-06T01:41:05.076999+0000","last_clean":"2023-10-06T01:41:05.076999+0000","last_became_active":"2023-10-06T01:41:05.072570+0000","last_became_peered":"2023-10-06T01:41:05.072570+0000","last_unstale":"2023-10-06T01:41:05.076999+0000","last_undegraded":"2023-10-06T01:41:05.076999+0000","last_fullsized":"2023-10-06T01:41:05.076999+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:05.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064241+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:41:05.064241+0000","last_peered":"2023-10-06T01:41:05.064241+0000","last_clean":"2023-10-06T01:41:05.064241+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:41:05.064241+0000","last_undegraded":"2023-10-06T01:41:05.064241+0000","last_fullsized":"2023-10-06T01:41:05.064241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066774+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:41:05.066774+0000","last_peered":"2023-10-06T01:41:05.066774+0000","last_clean":"2023-10-06T01:41:05.066774+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:41:05.066774+0000","last_undegraded":"2023-10-06T01:41:05.066774+0000","last_fullsized":"2023-10-06T01:41:05.066774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077743+0000","last_change":"2023-10-06T01:41:05.075317+0000","last_active":"2023-10-06T01:41:05.077743+0000","last_peered":"2023-10-06T01:41:05.077743+0000","last_clean":"2023-10-06T01:41:05.077743+0000","last_became_active":"2023-10-06T01:41:05.075139+0000","last_became_peered":"2023-10-06T01:41:05.075139+0000","last_unstale":"2023-10-06T01:41:05.077743+0000","last_undegraded":"2023-10-06T01:41:05.077743+0000","last_fullsized":"2023-10-06T01:41:05.077743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062926+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:41:05.062926+0000","last_peered":"2023-10-06T01:41:05.062926+0000","last_clean":"2023-10-06T01:41:05.062926+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:41:05.062926+0000","last_undegraded":"2023-10-06T01:41:05.062926+0000","last_fullsized":"2023-10-06T01:41:05.062926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064314+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:41:05.064314+0000","last_peered":"2023-10-06T01:41:05.064314+0000","last_clean":"2023-10-06T01:41:05.064314+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:41:05.064314+0000","last_undegraded":"2023-10-06T01:41:05.064314+0000","last_fullsized":"2023-10-06T01:41:05.064314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074604+0000","last_change":"2023-10-06T01:41:05.069137+0000","last_active":"2023-10-06T01:41:05.074604+0000","last_peered":"2023-10-06T01:41:05.074604+0000","last_clean":"2023-10-06T01:41:05.074604+0000","last_became_active":"2023-10-06T01:41:05.068861+0000","last_became_peered":"2023-10-06T01:41:05.068861+0000","last_unstale":"2023-10-06T01:41:05.074604+0000","last_undegraded":"2023-10-06T01:41:05.074604+0000","last_fullsized":"2023-10-06T01:41:05.074604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:54.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063296+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:41:05.063296+0000","last_peered":"2023-10-06T01:41:05.063296+0000","last_clean":"2023-10-06T01:41:05.063296+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:41:05.063296+0000","last_undegraded":"2023-10-06T01:41:05.063296+0000","last_fullsized":"2023-10-06T01:41:05.063296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063779+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:41:05.063779+0000","last_peered":"2023-10-06T01:41:05.063779+0000","last_clean":"2023-10-06T01:41:05.063779+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:41:05.063779+0000","last_undegraded":"2023-10-06T01:41:05.063779+0000","last_fullsized":"2023-10-06T01:41:05.063779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073651+0000","last_change":"2023-10-06T01:41:05.071706+0000","last_active":"2023-10-06T01:41:05.073651+0000","last_peered":"2023-10-06T01:41:05.073651+0000","last_clean":"2023-10-06T01:41:05.073651+0000","last_became_active":"2023-10-06T01:41:05.071432+0000","last_became_peered":"2023-10-06T01:41:05.071432+0000","last_unstale":"2023-10-06T01:41:05.073651+0000","last_undegraded":"2023-10-06T01:41:05.073651+0000","last_fullsized":"2023-10-06T01:41:05.073651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070266+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:41:05.070266+0000","last_peered":"2023-10-06T01:41:05.070266+0000","last_clean":"2023-10-06T01:41:05.070266+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:41:05.070266+0000","last_undegraded":"2023-10-06T01:41:05.070266+0000","last_fullsized":"2023-10-06T01:41:05.070266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067322+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:41:05.067322+0000","last_peered":"2023-10-06T01:41:05.067322+0000","last_clean":"2023-10-06T01:41:05.067322+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:41:05.067322+0000","last_undegraded":"2023-10-06T01:41:05.067322+0000","last_fullsized":"2023-10-06T01:41:05.067322+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070176+0000","last_change":"2023-10-06T01:41:05.066490+0000","last_active":"2023-10-06T01:41:05.070176+0000","last_peered":"2023-10-06T01:41:05.070176+0000","last_clean":"2023-10-06T01:41:05.070176+0000","last_became_active":"2023-10-06T01:41:05.065985+0000","last_became_peered":"2023-10-06T01:41:05.065985+0000","last_unstale":"2023-10-06T01:41:05.070176+0000","last_undegraded":"2023-10-06T01:41:05.070176+0000","last_fullsized":"2023-10-06T01:41:05.070176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067405+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:41:05.067405+0000","last_peered":"2023-10-06T01:41:05.067405+0000","last_clean":"2023-10-06T01:41:05.067405+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:41:05.067405+0000","last_undegraded":"2023-10-06T01:41:05.067405+0000","last_fullsized":"2023-10-06T01:41:05.067405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069275+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:41:05.069275+0000","last_peered":"2023-10-06T01:41:05.069275+0000","last_clean":"2023-10-06T01:41:05.069275+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:41:05.069275+0000","last_undegraded":"2023-10-06T01:41:05.069275+0000","last_fullsized":"2023-10-06T01:41:05.069275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074516+0000","last_change":"2023-10-06T01:41:05.071088+0000","last_active":"2023-10-06T01:41:05.074516+0000","last_peered":"2023-10-06T01:41:05.074516+0000","last_clean":"2023-10-06T01:41:05.074516+0000","last_became_active":"2023-10-06T01:41:05.070727+0000","last_became_peered":"2023-10-06T01:41:05.070727+0000","last_unstale":"2023-10-06T01:41:05.074516+0000","last_undegraded":"2023-10-06T01:41:05.074516+0000","last_fullsized":"2023-10-06T01:41:05.074516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068853+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:41:05.068853+0000","last_peered":"2023-10-06T01:41:05.068853+0000","last_clean":"2023-10-06T01:41:05.068853+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:41:05.068853+0000","last_undegraded":"2023-10-06T01:41:05.068853+0000","last_fullsized":"2023-10-06T01:41:05.068853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'22","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060056+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:41:05.060056+0000","last_peered":"2023-10-06T01:41:05.060056+0000","last_clean":"2023-10-06T01:41:05.060056+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:41:05.060056+0000","last_undegraded":"2023-10-06T01:41:05.060056+0000","last_fullsized":"2023-10-06T01:41:05.060056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077429+0000","last_change":"2023-10-06T01:41:05.072554+0000","last_active":"2023-10-06T01:41:05.077429+0000","last_peered":"2023-10-06T01:41:05.077429+0000","last_clean":"2023-10-06T01:41:05.077429+0000","last_became_active":"2023-10-06T01:41:05.072311+0000","last_became_peered":"2023-10-06T01:41:05.072311+0000","last_unstale":"2023-10-06T01:41:05.077429+0000","last_undegraded":"2023-10-06T01:41:05.077429+0000","last_fullsized":"2023-10-06T01:41:05.077429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062171+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:41:05.062171+0000","last_peered":"2023-10-06T01:41:05.062171+0000","last_clean":"2023-10-06T01:41:05.062171+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:41:05.062171+0000","last_undegraded":"2023-10-06T01:41:05.062171+0000","last_fullsized":"2023-10-06T01:41:05.062171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067767+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:41:05.067767+0000","last_peered":"2023-10-06T01:41:05.067767+0000","last_clean":"2023-10-06T01:41:05.067767+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:41:05.067767+0000","last_undegraded":"2023-10-06T01:41:05.067767+0000","last_fullsized":"2023-10-06T01:41:05.067767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074931+0000","last_change":"2023-10-06T01:41:05.072492+0000","last_active":"2023-10-06T01:41:05.074931+0000","last_peered":"2023-10-06T01:41:05.074931+0000","last_clean":"2023-10-06T01:41:05.074931+0000","last_became_active":"2023-10-06T01:41:05.072225+0000","last_became_peered":"2023-10-06T01:41:05.072225+0000","last_unstale":"2023-10-06T01:41:05.074931+0000","last_undegraded":"2023-10-06T01:41:05.074931+0000","last_fullsized":"2023-10-06T01:41:05.074931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.539127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063929+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:41:05.063929+0000","last_peered":"2023-10-06T01:41:05.063929+0000","last_clean":"2023-10-06T01:41:05.063929+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:41:05.063929+0000","last_undegraded":"2023-10-06T01:41:05.063929+0000","last_fullsized":"2023-10-06T01:41:05.063929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067895+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:41:05.067895+0000","last_peered":"2023-10-06T01:41:05.067895+0000","last_clean":"2023-10-06T01:41:05.067895+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:41:05.067895+0000","last_undegraded":"2023-10-06T01:41:05.067895+0000","last_fullsized":"2023-10-06T01:41:05.067895+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077494+0000","last_change":"2023-10-06T01:41:05.072597+0000","last_active":"2023-10-06T01:41:05.077494+0000","last_peered":"2023-10-06T01:41:05.077494+0000","last_clean":"2023-10-06T01:41:05.077494+0000","last_became_active":"2023-10-06T01:41:05.072379+0000","last_became_peered":"2023-10-06T01:41:05.072379+0000","last_unstale":"2023-10-06T01:41:05.077494+0000","last_undegraded":"2023-10-06T01:41:05.077494+0000","last_fullsized":"2023-10-06T01:41:05.077494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067386+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:41:05.067386+0000","last_peered":"2023-10-06T01:41:05.067386+0000","last_clean":"2023-10-06T01:41:05.067386+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:41:05.067386+0000","last_undegraded":"2023-10-06T01:41:05.067386+0000","last_fullsized":"2023-10-06T01:41:05.067386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066926+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:41:05.066926+0000","last_peered":"2023-10-06T01:41:05.066926+0000","last_clean":"2023-10-06T01:41:05.066926+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:41:05.066926+0000","last_undegraded":"2023-10-06T01:41:05.066926+0000","last_fullsized":"2023-10-06T01:41:05.066926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073491+0000","last_change":"2023-10-06T01:41:05.070833+0000","last_active":"2023-10-06T01:41:05.073491+0000","last_peered":"2023-10-06T01:41:05.073491+0000","last_clean":"2023-10-06T01:41:05.073491+0000","last_became_active":"2023-10-06T01:41:05.070486+0000","last_became_peered":"2023-10-06T01:41:05.070486+0000","last_unstale":"2023-10-06T01:41:05.073491+0000","last_undegraded":"2023-10-06T01:41:05.073491+0000","last_fullsized":"2023-10-06T01:41:05.073491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068137+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:41:05.068137+0000","last_peered":"2023-10-06T01:41:05.068137+0000","last_clean":"2023-10-06T01:41:05.068137+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:41:05.068137+0000","last_undegraded":"2023-10-06T01:41:05.068137+0000","last_fullsized":"2023-10-06T01:41:05.068137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064395+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:41:05.064395+0000","last_peered":"2023-10-06T01:41:05.064395+0000","last_clean":"2023-10-06T01:41:05.064395+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:41:05.064395+0000","last_undegraded":"2023-10-06T01:41:05.064395+0000","last_fullsized":"2023-10-06T01:41:05.064395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071482+0000","last_change":"2023-10-06T01:41:05.069668+0000","last_active":"2023-10-06T01:41:05.071482+0000","last_peered":"2023-10-06T01:41:05.071482+0000","last_clean":"2023-10-06T01:41:05.071482+0000","last_became_active":"2023-10-06T01:41:05.069033+0000","last_became_peered":"2023-10-06T01:41:05.069033+0000","last_unstale":"2023-10-06T01:41:05.071482+0000","last_undegraded":"2023-10-06T01:41:05.071482+0000","last_fullsized":"2023-10-06T01:41:05.071482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:14:32.941476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059936+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:41:05.059936+0000","last_peered":"2023-10-06T01:41:05.059936+0000","last_clean":"2023-10-06T01:41:05.059936+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:41:05.059936+0000","last_undegraded":"2023-10-06T01:41:05.059936+0000","last_fullsized":"2023-10-06T01:41:05.059936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060008+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:41:05.060008+0000","last_peered":"2023-10-06T01:41:05.060008+0000","last_clean":"2023-10-06T01:41:05.060008+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:41:05.060008+0000","last_undegraded":"2023-10-06T01:41:05.060008+0000","last_fullsized":"2023-10-06T01:41:05.060008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075650+0000","last_change":"2023-10-06T01:41:05.072868+0000","last_active":"2023-10-06T01:41:05.075650+0000","last_peered":"2023-10-06T01:41:05.075650+0000","last_clean":"2023-10-06T01:41:05.075650+0000","last_became_active":"2023-10-06T01:41:05.072621+0000","last_became_peered":"2023-10-06T01:41:05.072621+0000","last_unstale":"2023-10-06T01:41:05.075650+0000","last_undegraded":"2023-10-06T01:41:05.075650+0000","last_fullsized":"2023-10-06T01:41:05.075650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067784+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:41:05.067784+0000","last_peered":"2023-10-06T01:41:05.067784+0000","last_clean":"2023-10-06T01:41:05.067784+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:41:05.067784+0000","last_undegraded":"2023-10-06T01:41:05.067784+0000","last_fullsized":"2023-10-06T01:41:05.067784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'3","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063389+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:41:05.063389+0000","last_peered":"2023-10-06T01:41:05.063389+0000","last_clean":"2023-10-06T01:41:05.063389+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:41:05.063389+0000","last_undegraded":"2023-10-06T01:41:05.063389+0000","last_fullsized":"2023-10-06T01:41:05.063389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.080676+0000","last_change":"2023-10-06T01:41:05.073304+0000","last_active":"2023-10-06T01:41:05.080676+0000","last_peered":"2023-10-06T01:41:05.080676+0000","last_clean":"2023-10-06T01:41:05.080676+0000","last_became_active":"2023-10-06T01:41:05.073056+0000","last_became_peered":"2023-10-06T01:41:05.073056+0000","last_unstale":"2023-10-06T01:41:05.080676+0000","last_undegraded":"2023-10-06T01:41:05.080676+0000","last_fullsized":"2023-10-06T01:41:05.080676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:14.536410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061740+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:41:05.061740+0000","last_peered":"2023-10-06T01:41:05.061740+0000","last_clean":"2023-10-06T01:41:05.061740+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:41:05.061740+0000","last_undegraded":"2023-10-06T01:41:05.061740+0000","last_fullsized":"2023-10-06T01:41:05.061740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'2","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067860+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:41:05.067860+0000","last_peered":"2023-10-06T01:41:05.067860+0000","last_clean":"2023-10-06T01:41:05.067860+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:41:05.067860+0000","last_undegraded":"2023-10-06T01:41:05.067860+0000","last_fullsized":"2023-10-06T01:41:05.067860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068813+0000","last_change":"2023-10-06T01:41:05.066269+0000","last_active":"2023-10-06T01:41:05.068813+0000","last_peered":"2023-10-06T01:41:05.068813+0000","last_clean":"2023-10-06T01:41:05.068813+0000","last_became_active":"2023-10-06T01:41:05.065979+0000","last_became_peered":"2023-10-06T01:41:05.065979+0000","last_unstale":"2023-10-06T01:41:05.068813+0000","last_undegraded":"2023-10-06T01:41:05.068813+0000","last_fullsized":"2023-10-06T01:41:05.068813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062811+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:41:05.062811+0000","last_peered":"2023-10-06T01:41:05.062811+0000","last_clean":"2023-10-06T01:41:05.062811+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:41:05.062811+0000","last_undegraded":"2023-10-06T01:41:05.062811+0000","last_fullsized":"2023-10-06T01:41:05.062811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067544+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:41:05.067544+0000","last_peered":"2023-10-06T01:41:05.067544+0000","last_clean":"2023-10-06T01:41:05.067544+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:41:05.067544+0000","last_undegraded":"2023-10-06T01:41:05.067544+0000","last_fullsized":"2023-10-06T01:41:05.067544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070058+0000","last_change":"2023-10-06T01:41:05.064865+0000","last_active":"2023-10-06T01:41:05.070058+0000","last_peered":"2023-10-06T01:41:05.070058+0000","last_clean":"2023-10-06T01:41:05.070058+0000","last_became_active":"2023-10-06T01:41:05.064644+0000","last_became_peered":"2023-10-06T01:41:05.064644+0000","last_unstale":"2023-10-06T01:41:05.070058+0000","last_undegraded":"2023-10-06T01:41:05.070058+0000","last_fullsized":"2023-10-06T01:41:05.070058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:27.127112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067040+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:41:05.067040+0000","last_peered":"2023-10-06T01:41:05.067040+0000","last_clean":"2023-10-06T01:41:05.067040+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:41:05.067040+0000","last_undegraded":"2023-10-06T01:41:05.067040+0000","last_fullsized":"2023-10-06T01:41:05.067040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'4","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067848+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:41:05.067848+0000","last_peered":"2023-10-06T01:41:05.067848+0000","last_clean":"2023-10-06T01:41:05.067848+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:41:05.067848+0000","last_undegraded":"2023-10-06T01:41:05.067848+0000","last_fullsized":"2023-10-06T01:41:05.067848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074693+0000","last_change":"2023-10-06T01:41:05.069995+0000","last_active":"2023-10-06T01:41:05.074693+0000","last_peered":"2023-10-06T01:41:05.074693+0000","last_clean":"2023-10-06T01:41:05.074693+0000","last_became_active":"2023-10-06T01:41:05.069709+0000","last_became_peered":"2023-10-06T01:41:05.069709+0000","last_unstale":"2023-10-06T01:41:05.074693+0000","last_undegraded":"2023-10-06T01:41:05.074693+0000","last_fullsized":"2023-10-06T01:41:05.074693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:50.555907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062679+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:41:05.062679+0000","last_peered":"2023-10-06T01:41:05.062679+0000","last_clean":"2023-10-06T01:41:05.062679+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:41:05.062679+0000","last_undegraded":"2023-10-06T01:41:05.062679+0000","last_fullsized":"2023-10-06T01:41:05.062679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066603+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:41:05.066603+0000","last_peered":"2023-10-06T01:41:05.066603+0000","last_clean":"2023-10-06T01:41:05.066603+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:41:05.066603+0000","last_undegraded":"2023-10-06T01:41:05.066603+0000","last_fullsized":"2023-10-06T01:41:05.066603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077363+0000","last_change":"2023-10-06T01:41:05.071858+0000","last_active":"2023-10-06T01:41:05.077363+0000","last_peered":"2023-10-06T01:41:05.077363+0000","last_clean":"2023-10-06T01:41:05.077363+0000","last_became_active":"2023-10-06T01:41:05.071590+0000","last_became_peered":"2023-10-06T01:41:05.071590+0000","last_unstale":"2023-10-06T01:41:05.077363+0000","last_undegraded":"2023-10-06T01:41:05.077363+0000","last_fullsized":"2023-10-06T01:41:05.077363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:39.551110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059831+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:41:05.059831+0000","last_peered":"2023-10-06T01:41:05.059831+0000","last_clean":"2023-10-06T01:41:05.059831+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:41:05.059831+0000","last_undegraded":"2023-10-06T01:41:05.059831+0000","last_fullsized":"2023-10-06T01:41:05.059831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063540+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:41:05.063540+0000","last_peered":"2023-10-06T01:41:05.063540+0000","last_clean":"2023-10-06T01:41:05.063540+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:41:05.063540+0000","last_undegraded":"2023-10-06T01:41:05.063540+0000","last_fullsized":"2023-10-06T01:41:05.063540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069555+0000","last_change":"2023-10-06T01:41:05.068429+0000","last_active":"2023-10-06T01:41:05.069555+0000","last_peered":"2023-10-06T01:41:05.069555+0000","last_clean":"2023-10-06T01:41:05.069555+0000","last_became_active":"2023-10-06T01:41:05.068180+0000","last_became_peered":"2023-10-06T01:41:05.068180+0000","last_unstale":"2023-10-06T01:41:05.069555+0000","last_undegraded":"2023-10-06T01:41:05.069555+0000","last_fullsized":"2023-10-06T01:41:05.069555+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"36'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063653+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:41:05.063653+0000","last_peered":"2023-10-06T01:41:05.063653+0000","last_clean":"2023-10-06T01:41:05.063653+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:41:05.063653+0000","last_undegraded":"2023-10-06T01:41:05.063653+0000","last_fullsized":"2023-10-06T01:41:05.063653+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067505+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:41:05.067505+0000","last_peered":"2023-10-06T01:41:05.067505+0000","last_clean":"2023-10-06T01:41:05.067505+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:41:05.067505+0000","last_undegraded":"2023-10-06T01:41:05.067505+0000","last_fullsized":"2023-10-06T01:41:05.067505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077459+0000","last_change":"2023-10-06T01:41:05.071544+0000","last_active":"2023-10-06T01:41:05.077459+0000","last_peered":"2023-10-06T01:41:05.077459+0000","last_clean":"2023-10-06T01:41:05.077459+0000","last_became_active":"2023-10-06T01:41:05.071300+0000","last_became_peered":"2023-10-06T01:41:05.071300+0000","last_unstale":"2023-10-06T01:41:05.077459+0000","last_undegraded":"2023-10-06T01:41:05.077459+0000","last_fullsized":"2023-10-06T01:41:05.077459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067681+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:41:05.067681+0000","last_peered":"2023-10-06T01:41:05.067681+0000","last_clean":"2023-10-06T01:41:05.067681+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:41:05.067681+0000","last_undegraded":"2023-10-06T01:41:05.067681+0000","last_fullsized":"2023-10-06T01:41:05.067681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063996+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:41:05.063996+0000","last_peered":"2023-10-06T01:41:05.063996+0000","last_clean":"2023-10-06T01:41:05.063996+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:41:05.063996+0000","last_undegraded":"2023-10-06T01:41:05.063996+0000","last_fullsized":"2023-10-06T01:41:05.063996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062842+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:05.062842+0000","last_peered":"2023-10-06T01:41:05.062842+0000","last_clean":"2023-10-06T01:41:05.062842+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:05.062842+0000","last_undegraded":"2023-10-06T01:41:05.062842+0000","last_fullsized":"2023-10-06T01:41:05.062842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074573+0000","last_change":"2023-10-06T01:41:05.070827+0000","last_active":"2023-10-06T01:41:05.074573+0000","last_peered":"2023-10-06T01:41:05.074573+0000","last_clean":"2023-10-06T01:41:05.074573+0000","last_became_active":"2023-10-06T01:41:05.070649+0000","last_became_peered":"2023-10-06T01:41:05.070649+0000","last_unstale":"2023-10-06T01:41:05.074573+0000","last_undegraded":"2023-10-06T01:41:05.074573+0000","last_fullsized":"2023-10-06T01:41:05.074573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:26.468333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067196+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:41:05.067196+0000","last_peered":"2023-10-06T01:41:05.067196+0000","last_clean":"2023-10-06T01:41:05.067196+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:41:05.067196+0000","last_undegraded":"2023-10-06T01:41:05.067196+0000","last_fullsized":"2023-10-06T01:41:05.067196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067890+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:41:05.067890+0000","last_peered":"2023-10-06T01:41:05.067890+0000","last_clean":"2023-10-06T01:41:05.067890+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:41:05.067890+0000","last_undegraded":"2023-10-06T01:41:05.067890+0000","last_fullsized":"2023-10-06T01:41:05.067890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067611+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:05.067611+0000","last_peered":"2023-10-06T01:41:05.067611+0000","last_clean":"2023-10-06T01:41:05.067611+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:05.067611+0000","last_undegraded":"2023-10-06T01:41:05.067611+0000","last_fullsized":"2023-10-06T01:41:05.067611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071384+0000","last_change":"2023-10-06T01:41:05.068635+0000","last_active":"2023-10-06T01:41:05.071384+0000","last_peered":"2023-10-06T01:41:05.071384+0000","last_clean":"2023-10-06T01:41:05.071384+0000","last_became_active":"2023-10-06T01:41:05.068364+0000","last_became_peered":"2023-10-06T01:41:05.068364+0000","last_unstale":"2023-10-06T01:41:05.071384+0000","last_undegraded":"2023-10-06T01:41:05.071384+0000","last_fullsized":"2023-10-06T01:41:05.071384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:53.027266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069163+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:41:05.069163+0000","last_peered":"2023-10-06T01:41:05.069163+0000","last_clean":"2023-10-06T01:41:05.069163+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:41:05.069163+0000","last_undegraded":"2023-10-06T01:41:05.069163+0000","last_fullsized":"2023-10-06T01:41:05.069163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062581+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:41:05.062581+0000","last_peered":"2023-10-06T01:41:05.062581+0000","last_clean":"2023-10-06T01:41:05.062581+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:41:05.062581+0000","last_undegraded":"2023-10-06T01:41:05.062581+0000","last_fullsized":"2023-10-06T01:41:05.062581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067978+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:05.067978+0000","last_peered":"2023-10-06T01:41:05.067978+0000","last_clean":"2023-10-06T01:41:05.067978+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:05.067978+0000","last_undegraded":"2023-10-06T01:41:05.067978+0000","last_fullsized":"2023-10-06T01:41:05.067978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075630+0000","last_change":"2023-10-06T01:41:05.072823+0000","last_active":"2023-10-06T01:41:05.075630+0000","last_peered":"2023-10-06T01:41:05.075630+0000","last_clean":"2023-10-06T01:41:05.075630+0000","last_became_active":"2023-10-06T01:41:05.072511+0000","last_became_peered":"2023-10-06T01:41:05.072511+0000","last_unstale":"2023-10-06T01:41:05.075630+0000","last_undegraded":"2023-10-06T01:41:05.075630+0000","last_fullsized":"2023-10-06T01:41:05.075630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064300+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:41:05.064300+0000","last_peered":"2023-10-06T01:41:05.064300+0000","last_clean":"2023-10-06T01:41:05.064300+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:41:05.064300+0000","last_undegraded":"2023-10-06T01:41:05.064300+0000","last_fullsized":"2023-10-06T01:41:05.064300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069409+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:41:05.069409+0000","last_peered":"2023-10-06T01:41:05.069409+0000","last_clean":"2023-10-06T01:41:05.069409+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:41:05.069409+0000","last_undegraded":"2023-10-06T01:41:05.069409+0000","last_fullsized":"2023-10-06T01:41:05.069409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067997+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:05.067997+0000","last_peered":"2023-10-06T01:41:05.067997+0000","last_clean":"2023-10-06T01:41:05.067997+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:05.067997+0000","last_undegraded":"2023-10-06T01:41:05.067997+0000","last_fullsized":"2023-10-06T01:41:05.067997+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075181+0000","last_change":"2023-10-06T01:41:05.072739+0000","last_active":"2023-10-06T01:41:05.075181+0000","last_peered":"2023-10-06T01:41:05.075181+0000","last_clean":"2023-10-06T01:41:05.075181+0000","last_became_active":"2023-10-06T01:41:05.072486+0000","last_became_peered":"2023-10-06T01:41:05.072486+0000","last_unstale":"2023-10-06T01:41:05.075181+0000","last_undegraded":"2023-10-06T01:41:05.075181+0000","last_fullsized":"2023-10-06T01:41:05.075181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:44.012384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067237+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:41:05.067237+0000","last_peered":"2023-10-06T01:41:05.067237+0000","last_clean":"2023-10-06T01:41:05.067237+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:41:05.067237+0000","last_undegraded":"2023-10-06T01:41:05.067237+0000","last_fullsized":"2023-10-06T01:41:05.067237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068333+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:41:05.068333+0000","last_peered":"2023-10-06T01:41:05.068333+0000","last_clean":"2023-10-06T01:41:05.068333+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:41:05.068333+0000","last_undegraded":"2023-10-06T01:41:05.068333+0000","last_fullsized":"2023-10-06T01:41:05.068333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067888+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:05.067888+0000","last_peered":"2023-10-06T01:41:05.067888+0000","last_clean":"2023-10-06T01:41:05.067888+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:05.067888+0000","last_undegraded":"2023-10-06T01:41:05.067888+0000","last_fullsized":"2023-10-06T01:41:05.067888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074585+0000","last_change":"2023-10-06T01:41:05.071521+0000","last_active":"2023-10-06T01:41:05.074585+0000","last_peered":"2023-10-06T01:41:05.074585+0000","last_clean":"2023-10-06T01:41:05.074585+0000","last_became_active":"2023-10-06T01:41:05.071272+0000","last_became_peered":"2023-10-06T01:41:05.071272+0000","last_unstale":"2023-10-06T01:41:05.074585+0000","last_undegraded":"2023-10-06T01:41:05.074585+0000","last_fullsized":"2023-10-06T01:41:05.074585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.431329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'3","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068093+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:41:05.068093+0000","last_peered":"2023-10-06T01:41:05.068093+0000","last_clean":"2023-10-06T01:41:05.068093+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:41:05.068093+0000","last_undegraded":"2023-10-06T01:41:05.068093+0000","last_fullsized":"2023-10-06T01:41:05.068093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066856+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:05.066856+0000","last_peered":"2023-10-06T01:41:05.066856+0000","last_clean":"2023-10-06T01:41:05.066856+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:05.066856+0000","last_undegraded":"2023-10-06T01:41:05.066856+0000","last_fullsized":"2023-10-06T01:41:05.066856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077722+0000","last_change":"2023-10-06T01:41:05.073694+0000","last_active":"2023-10-06T01:41:05.077722+0000","last_peered":"2023-10-06T01:41:05.077722+0000","last_clean":"2023-10-06T01:41:05.077722+0000","last_became_active":"2023-10-06T01:41:05.073261+0000","last_became_peered":"2023-10-06T01:41:05.073261+0000","last_unstale":"2023-10-06T01:41:05.077722+0000","last_undegraded":"2023-10-06T01:41:05.077722+0000","last_fullsized":"2023-10-06T01:41:05.077722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066331+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:41:05.066331+0000","last_peered":"2023-10-06T01:41:05.066331+0000","last_clean":"2023-10-06T01:41:05.066331+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:41:05.066331+0000","last_undegraded":"2023-10-06T01:41:05.066331+0000","last_fullsized":"2023-10-06T01:41:05.066331+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062956+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:41:05.062956+0000","last_peered":"2023-10-06T01:41:05.062956+0000","last_clean":"2023-10-06T01:41:05.062956+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:41:05.062956+0000","last_undegraded":"2023-10-06T01:41:05.062956+0000","last_fullsized":"2023-10-06T01:41:05.062956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062529+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:05.062529+0000","last_peered":"2023-10-06T01:41:05.062529+0000","last_clean":"2023-10-06T01:41:05.062529+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:05.062529+0000","last_undegraded":"2023-10-06T01:41:05.062529+0000","last_fullsized":"2023-10-06T01:41:05.062529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.072041+0000","last_change":"2023-10-06T01:41:05.070184+0000","last_active":"2023-10-06T01:41:05.072041+0000","last_peered":"2023-10-06T01:41:05.072041+0000","last_clean":"2023-10-06T01:41:05.072041+0000","last_became_active":"2023-10-06T01:41:05.069937+0000","last_became_peered":"2023-10-06T01:41:05.069937+0000","last_unstale":"2023-10-06T01:41:05.072041+0000","last_undegraded":"2023-10-06T01:41:05.072041+0000","last_fullsized":"2023-10-06T01:41:05.072041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"36'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062666+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:41:05.062666+0000","last_peered":"2023-10-06T01:41:05.062666+0000","last_clean":"2023-10-06T01:41:05.062666+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:41:05.062666+0000","last_undegraded":"2023-10-06T01:41:05.062666+0000","last_fullsized":"2023-10-06T01:41:05.062666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064473+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:41:05.064473+0000","last_peered":"2023-10-06T01:41:05.064473+0000","last_clean":"2023-10-06T01:41:05.064473+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:41:05.064473+0000","last_undegraded":"2023-10-06T01:41:05.064473+0000","last_fullsized":"2023-10-06T01:41:05.064473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062127+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:05.062127+0000","last_peered":"2023-10-06T01:41:05.062127+0000","last_clean":"2023-10-06T01:41:05.062127+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:05.062127+0000","last_undegraded":"2023-10-06T01:41:05.062127+0000","last_fullsized":"2023-10-06T01:41:05.062127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.88},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.755},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.309}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.621}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":237748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.312}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":239101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41199999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.622},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:09.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T01:41:10.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.004+0000 7fae48e1c700 1 Processor -- start 2023-10-06T01:41:10.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.004+0000 7fae48e1c700 1 -- start start 2023-10-06T01:41:10.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae48e1c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae441595a0 0x7fae440aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae48e1c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae48e1c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440b0ca0 con 0x7fae440b49e0 2023-10-06T01:41:10.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae48e1c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440b0f80 con 0x7fae441595a0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae48e1c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b1260 con 0x7fae440b07a0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae441595a0 0x7fae440aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae441595a0 0x7fae440aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55168/0 (socket says 172.21.15.98:55168) 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 -- 172.21.15.98:0/205233808 learned_addr learned my addr 172.21.15.98:0/205233808 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae42d9d700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 -- 172.21.15.98:0/205233808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 msgr2=0x7fae440b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 -- 172.21.15.98:0/205233808 >> v1:172.21.15.184:6789/0 conn(0x7fae440b49e0 legacy=0x7fae440aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.005+0000 7fae4259c700 1 -- 172.21.15.98:0/205233808 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440b1540 con 0x7fae441595a0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae4259c700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440aa570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fae340099c0 tx=0x7fae34009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114655a2fc277d7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae41d9b700 1 -- 172.21.15.98:0/205233808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34005c10 con 0x7fae441595a0 2023-10-06T01:41:10.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae41d9b700 1 -- 172.21.15.98:0/205233808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34005d70 con 0x7fae441595a0 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae41d9b700 1 -- 172.21.15.98:0/205233808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400b840 con 0x7fae441595a0 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 -- 172.21.15.98:0/205233808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 msgr2=0x7fae440aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440aa570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fae340099c0 tx=0x7fae34009220 comp rx=0 tx=0).stop 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 -- 172.21.15.98:0/205233808 shutdown_connections 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 --2- 172.21.15.98:0/205233808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440aa570 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 -- 172.21.15.98:0/205233808 >> 172.21.15.98:0/205233808 conn(0x7fae44099830 msgr2=0x7fae4406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 -- 172.21.15.98:0/205233808 shutdown_connections 2023-10-06T01:41:10.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.006+0000 7fae48e1c700 1 -- 172.21.15.98:0/205233808 wait complete. 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 Processor -- start 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 -- start start 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 0x7fae440b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440abf20 con 0x7fae441595a0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4416e570 con 0x7fae440b49e0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae48e1c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440ab5f0 con 0x7fae440b07a0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae42d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae42d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55182/0 (socket says 172.21.15.98:55182) 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.008+0000 7fae42d9d700 1 -- 172.21.15.98:0/4001939441 learned_addr learned my addr 172.21.15.98:0/4001939441 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae4259c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae39d9b700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 0x7fae440b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 -- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 msgr2=0x7fae440aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 -- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 msgr2=0x7fae440b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 0x7fae440b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 -- 172.21.15.98:0/4001939441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009480 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae39d9b700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 0x7fae440b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae42d9d700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fae3c007f30 tx=0x7fae3c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00ec40 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c00eda0 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440ab8d0 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c014670 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.009+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440af490 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.010+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae440ac0d0 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.010+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae3c014a20 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.011+0000 7fae3affd700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.011+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fae3c0850e0 con 0x7fae441595a0 2023-10-06T01:41:10.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.011+0000 7fae4259c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.011+0000 7fae4259c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae34009260 tx=0x7fae34016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.015+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae3c051e90 con 0x7fae441595a0 2023-10-06T01:41:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.279+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fae440af620 con 0x7fae441595a0 2023-10-06T01:41:10.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.280+0000 7fae3affd700 1 -- 172.21.15.98:0/4001939441 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1572 (secure 0 0 0) 0x7fae3c009a20 con 0x7fae441595a0 2023-10-06T01:41:10.282 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:10.282 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","health":{"status":"HEALTH_ERR","checks":{"FS_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"2 filesystems are degraded","count":2},"muted":false},"MDS_ALL_DOWN":{"severity":"HEALTH_ERR","summary":{"message":"2 filesystems are offline","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":104,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1696556377,"num_in_osds":8,"osd_in_since":1696556368,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":137}],"num_pgs":137,"num_pools":5,"num_objects":4,"data_bytes":6881824,"bytes_used":2453245952,"bytes_avail":770640867328,"bytes_total":773094113280},"fsmap":{"epoch":23,"id":2,"up":0,"in":1,"max":1,"id":3,"up":0,"in":1,"max":1,"by_rank":[],"up:standby":4,"failed":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-06T01:40:55.799799+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 msgr2=0x7fae240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae34009260 tx=0x7fae34016000 comp rx=0 tx=0).stop 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 msgr2=0x7fae440b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fae3c007f30 tx=0x7fae3c0098e0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 shutdown_connections 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae441595a0 0x7fae440b20d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae440b07a0 0x7fae440aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 --2- 172.21.15.98:0/4001939441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae440b49e0 0x7fae440b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 >> 172.21.15.98:0/4001939441 conn(0x7fae44099830 msgr2=0x7fae44158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:10.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 shutdown_connections 2023-10-06T01:41:10.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.282+0000 7fae48e1c700 1 -- 172.21.15.98:0/4001939441 wait complete. 2023-10-06T01:41:10.295 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T01:41:10.295 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:10.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.408+0000 7f9c5af7e700 1 Processor -- start 2023-10-06T01:41:10.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.408+0000 7f9c5af7e700 1 -- start start 2023-10-06T01:41:10.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5af7e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9c541595a0 0x7f9c540aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5af7e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9c540b07a0 0x7f9c540b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.412 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5af7e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c540b0ca0 con 0x7f9c540b49e0 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5af7e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c540b0f80 con 0x7f9c540b07a0 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5af7e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c540b1260 con 0x7f9c541595a0 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9c540b07a0 0x7f9c540b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c53fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9c540b49e0 0x7f9c540aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55344/0 (socket says 172.21.15.98:55344) 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9c540b07a0 0x7f9c540b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55188/0 (socket says 172.21.15.98:55188) 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c53fff700 1 -- 172.21.15.98:0/3293685883 learned_addr learned my addr 172.21.15.98:0/3293685883 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 -- 172.21.15.98:0/3293685883 >> v2:172.21.15.184:3300/0 conn(0x7f9c541595a0 msgr2=0x7f9c540aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c58d1a700 1 --2- 172.21.15.98:0/3293685883 >> v2:172.21.15.184:3300/0 conn(0x7f9c541595a0 0x7f9c540aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 --2- 172.21.15.98:0/3293685883 >> v2:172.21.15.184:3300/0 conn(0x7f9c541595a0 0x7f9c540aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 -- 172.21.15.98:0/3293685883 >> v1:172.21.15.98:6789/0 conn(0x7f9c540b49e0 legacy=0x7f9c540aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c537fe700 1 -- 172.21.15.98:0/3293685883 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693120164 0 0) 0x7f9c540b0ca0 con 0x7f9c540b49e0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 -- 172.21.15.98:0/3293685883 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c540b1540 con 0x7f9c540b07a0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.409+0000 7f9c5951b700 1 --2- 172.21.15.98:0/3293685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c540b1c00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0098e0 tx=0x7f9c3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d83c749b7133bca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c537fe700 1 -- 172.21.15.98:0/3293685883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c009e80 con 0x7f9c540b07a0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c537fe700 1 -- 172.21.15.98:0/3293685883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00b820 con 0x7f9c540b07a0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c537fe700 1 -- 172.21.15.98:0/3293685883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bba0 con 0x7f9c540b07a0 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 -- 172.21.15.98:0/3293685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 msgr2=0x7f9c540b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/3293685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c540b1c00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0098e0 tx=0x7f9c3c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 -- 172.21.15.98:0/3293685883 shutdown_connections 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/3293685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c540b1c00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/3293685883 >> v2:172.21.15.184:3300/0 conn(0x7f9c541595a0 0x7f9c540aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 -- 172.21.15.98:0/3293685883 >> 172.21.15.98:0/3293685883 conn(0x7f9c54099830 msgr2=0x7f9c5406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 -- 172.21.15.98:0/3293685883 shutdown_connections 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.410+0000 7f9c5af7e700 1 -- 172.21.15.98:0/3293685883 wait complete. 2023-10-06T01:41:10.415 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.411+0000 7f9c5af7e700 1 Processor -- start 2023-10-06T01:41:10.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.411+0000 7f9c5af7e700 1 -- start start 2023-10-06T01:41:10.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 0x7f9c5415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c541595a0 0x7f9c5409a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c541697c0 con 0x7f9c540b07a0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5415ab10 con 0x7f9c540b49e0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5af7e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5415adf0 con 0x7f9c541595a0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55204/0 (socket says 172.21.15.98:55204) 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 -- 172.21.15.98:0/1597217515 learned_addr learned my addr 172.21.15.98:0/1597217515 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c53fff700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 0x7f9c5415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c5951b700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c541595a0 0x7f9c5409a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 -- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c541595a0 msgr2=0x7f9c5409a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c541595a0 0x7f9c5409a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.418 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 -- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 msgr2=0x7f9c5415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 0x7f9c5415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c58d1a700 1 -- 172.21.15.98:0/1597217515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009420 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.412+0000 7f9c53fff700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 0x7f9c5415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c58d1a700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ceb0 tx=0x7f9c4c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00adf0 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c00f450 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5415b0d0 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00f7f0 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.413+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5415d010 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.414+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54064ad0 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.414+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c4c003020 con 0x7f9c540b07a0 2023-10-06T01:41:10.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.415+0000 7f9c50ff9700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 0x7f9c340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:10.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.415+0000 7f9c53fff700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 0x7f9c340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:10.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.415+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f9c4c013070 con 0x7f9c540b07a0 2023-10-06T01:41:10.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.415+0000 7f9c53fff700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 0x7f9c340693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c5409dee0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:10.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.419+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c4c052140 con 0x7f9c540b07a0 2023-10-06T01:41:10.597 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.596+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c540b1820 con 0x7f9c34066f30 2023-10-06T01:41:10.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.606+0000 7f9c50ff9700 1 -- 172.21.15.98:0/1597217515 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+392321 (secure 0 0 0) 0x7f9c540b1820 con 0x7f9c34066f30 2023-10-06T01:41:10.612 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 msgr2=0x7f9c340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 0x7f9c340693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c5409dee0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2023-10-06T01:41:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 msgr2=0x7f9c54158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ceb0 tx=0x7f9c4c007860 comp rx=0 tx=0).stop 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 shutdown_connections 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9c34066f30 0x7f9c340693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c541595a0 0x7f9c5409a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c540b07a0 0x7f9c54158b90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.609+0000 7f9c5af7e700 1 --2- 172.21.15.98:0/1597217515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c540b49e0 0x7f9c5415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.610+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 >> 172.21.15.98:0/1597217515 conn(0x7f9c54099830 msgr2=0x7f9c540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.610+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 shutdown_connections 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:10.610+0000 7f9c5af7e700 1 -- 172.21.15.98:0/1597217515 wait complete. 2023-10-06T01:41:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:10.623 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-06T01:41:09.802971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":255,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395748,"kb_used_data":17444,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578972,"statfs":{"total":773094113280,"available":770640867328,"internally_reserved":0,"allocated":17862656,"data_stored":15675103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001235"},"pg_stats":[{"pgid":"7.3f","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070953+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:41:05.070953+0000","last_peered":"2023-10-06T01:41:05.070953+0000","last_clean":"2023-10-06T01:41:05.070953+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:41:05.070953+0000","last_undegraded":"2023-10-06T01:41:05.070953+0000","last_fullsized":"2023-10-06T01:41:05.070953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068055+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:41:05.068055+0000","last_peered":"2023-10-06T01:41:05.068055+0000","last_clean":"2023-10-06T01:41:05.068055+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:41:05.068055+0000","last_undegraded":"2023-10-06T01:41:05.068055+0000","last_fullsized":"2023-10-06T01:41:05.068055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062731+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:41:05.062731+0000","last_peered":"2023-10-06T01:41:05.062731+0000","last_clean":"2023-10-06T01:41:05.062731+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:41:05.062731+0000","last_undegraded":"2023-10-06T01:41:05.062731+0000","last_fullsized":"2023-10-06T01:41:05.062731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064200+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:41:05.064200+0000","last_peered":"2023-10-06T01:41:05.064200+0000","last_clean":"2023-10-06T01:41:05.064200+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:41:05.064200+0000","last_undegraded":"2023-10-06T01:41:05.064200+0000","last_fullsized":"2023-10-06T01:41:05.064200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067188+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:41:05.067188+0000","last_peered":"2023-10-06T01:41:05.067188+0000","last_clean":"2023-10-06T01:41:05.067188+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:41:05.067188+0000","last_undegraded":"2023-10-06T01:41:05.067188+0000","last_fullsized":"2023-10-06T01:41:05.067188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067168+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:41:05.067168+0000","last_peered":"2023-10-06T01:41:05.067168+0000","last_clean":"2023-10-06T01:41:05.067168+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:41:05.067168+0000","last_undegraded":"2023-10-06T01:41:05.067168+0000","last_fullsized":"2023-10-06T01:41:05.067168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068070+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:41:05.068070+0000","last_peered":"2023-10-06T01:41:05.068070+0000","last_clean":"2023-10-06T01:41:05.068070+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:41:05.068070+0000","last_undegraded":"2023-10-06T01:41:05.068070+0000","last_fullsized":"2023-10-06T01:41:05.068070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063265+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:41:05.063265+0000","last_peered":"2023-10-06T01:41:05.063265+0000","last_clean":"2023-10-06T01:41:05.063265+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:41:05.063265+0000","last_undegraded":"2023-10-06T01:41:05.063265+0000","last_fullsized":"2023-10-06T01:41:05.063265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"36'11","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067676+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:41:05.067676+0000","last_peered":"2023-10-06T01:41:05.067676+0000","last_clean":"2023-10-06T01:41:05.067676+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:41:05.067676+0000","last_undegraded":"2023-10-06T01:41:05.067676+0000","last_fullsized":"2023-10-06T01:41:05.067676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064867+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:41:05.064867+0000","last_peered":"2023-10-06T01:41:05.064867+0000","last_clean":"2023-10-06T01:41:05.064867+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:41:05.064867+0000","last_undegraded":"2023-10-06T01:41:05.064867+0000","last_fullsized":"2023-10-06T01:41:05.064867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064219+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:41:05.064219+0000","last_peered":"2023-10-06T01:41:05.064219+0000","last_clean":"2023-10-06T01:41:05.064219+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:41:05.064219+0000","last_undegraded":"2023-10-06T01:41:05.064219+0000","last_fullsized":"2023-10-06T01:41:05.064219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066867+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:41:05.066867+0000","last_peered":"2023-10-06T01:41:05.066867+0000","last_clean":"2023-10-06T01:41:05.066867+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:41:05.066867+0000","last_undegraded":"2023-10-06T01:41:05.066867+0000","last_fullsized":"2023-10-06T01:41:05.066867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"36'10","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059981+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:41:05.059981+0000","last_peered":"2023-10-06T01:41:05.059981+0000","last_clean":"2023-10-06T01:41:05.059981+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:41:05.059981+0000","last_undegraded":"2023-10-06T01:41:05.059981+0000","last_fullsized":"2023-10-06T01:41:05.059981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063679+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:41:05.063679+0000","last_peered":"2023-10-06T01:41:05.063679+0000","last_clean":"2023-10-06T01:41:05.063679+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:41:05.063679+0000","last_undegraded":"2023-10-06T01:41:05.063679+0000","last_fullsized":"2023-10-06T01:41:05.063679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066663+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:41:05.066663+0000","last_peered":"2023-10-06T01:41:05.066663+0000","last_clean":"2023-10-06T01:41:05.066663+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:41:05.066663+0000","last_undegraded":"2023-10-06T01:41:05.066663+0000","last_fullsized":"2023-10-06T01:41:05.066663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061713+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:41:05.061713+0000","last_peered":"2023-10-06T01:41:05.061713+0000","last_clean":"2023-10-06T01:41:05.061713+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:41:05.061713+0000","last_undegraded":"2023-10-06T01:41:05.061713+0000","last_fullsized":"2023-10-06T01:41:05.061713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063591+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:41:05.063591+0000","last_peered":"2023-10-06T01:41:05.063591+0000","last_clean":"2023-10-06T01:41:05.063591+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:41:05.063591+0000","last_undegraded":"2023-10-06T01:41:05.063591+0000","last_fullsized":"2023-10-06T01:41:05.063591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062553+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:41:05.062553+0000","last_peered":"2023-10-06T01:41:05.062553+0000","last_clean":"2023-10-06T01:41:05.062553+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:41:05.062553+0000","last_undegraded":"2023-10-06T01:41:05.062553+0000","last_fullsized":"2023-10-06T01:41:05.062553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062556+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:41:05.062556+0000","last_peered":"2023-10-06T01:41:05.062556+0000","last_clean":"2023-10-06T01:41:05.062556+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:41:05.062556+0000","last_undegraded":"2023-10-06T01:41:05.062556+0000","last_fullsized":"2023-10-06T01:41:05.062556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059961+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:41:05.059961+0000","last_peered":"2023-10-06T01:41:05.059961+0000","last_clean":"2023-10-06T01:41:05.059961+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:41:05.059961+0000","last_undegraded":"2023-10-06T01:41:05.059961+0000","last_fullsized":"2023-10-06T01:41:05.059961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067584+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:41:05.067584+0000","last_peered":"2023-10-06T01:41:05.067584+0000","last_clean":"2023-10-06T01:41:05.067584+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:41:05.067584+0000","last_undegraded":"2023-10-06T01:41:05.067584+0000","last_fullsized":"2023-10-06T01:41:05.067584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063813+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:41:05.063813+0000","last_peered":"2023-10-06T01:41:05.063813+0000","last_clean":"2023-10-06T01:41:05.063813+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:41:05.063813+0000","last_undegraded":"2023-10-06T01:41:05.063813+0000","last_fullsized":"2023-10-06T01:41:05.063813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"36'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062072+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:41:05.062072+0000","last_peered":"2023-10-06T01:41:05.062072+0000","last_clean":"2023-10-06T01:41:05.062072+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:41:05.062072+0000","last_undegraded":"2023-10-06T01:41:05.062072+0000","last_fullsized":"2023-10-06T01:41:05.062072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"36'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059914+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:41:05.059914+0000","last_peered":"2023-10-06T01:41:05.059914+0000","last_clean":"2023-10-06T01:41:05.059914+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:41:05.059914+0000","last_undegraded":"2023-10-06T01:41:05.059914+0000","last_fullsized":"2023-10-06T01:41:05.059914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062784+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:41:05.062784+0000","last_peered":"2023-10-06T01:41:05.062784+0000","last_clean":"2023-10-06T01:41:05.062784+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:41:05.062784+0000","last_undegraded":"2023-10-06T01:41:05.062784+0000","last_fullsized":"2023-10-06T01:41:05.062784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067606+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:41:05.067606+0000","last_peered":"2023-10-06T01:41:05.067606+0000","last_clean":"2023-10-06T01:41:05.067606+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:41:05.067606+0000","last_undegraded":"2023-10-06T01:41:05.067606+0000","last_fullsized":"2023-10-06T01:41:05.067606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061786+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:41:05.061786+0000","last_peered":"2023-10-06T01:41:05.061786+0000","last_clean":"2023-10-06T01:41:05.061786+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:41:05.061786+0000","last_undegraded":"2023-10-06T01:41:05.061786+0000","last_fullsized":"2023-10-06T01:41:05.061786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063049+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:41:05.063049+0000","last_peered":"2023-10-06T01:41:05.063049+0000","last_clean":"2023-10-06T01:41:05.063049+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:41:05.063049+0000","last_undegraded":"2023-10-06T01:41:05.063049+0000","last_fullsized":"2023-10-06T01:41:05.063049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"36'12","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066771+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:41:05.066771+0000","last_peered":"2023-10-06T01:41:05.066771+0000","last_clean":"2023-10-06T01:41:05.066771+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:41:05.066771+0000","last_undegraded":"2023-10-06T01:41:05.066771+0000","last_fullsized":"2023-10-06T01:41:05.066771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059829+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:41:05.059829+0000","last_peered":"2023-10-06T01:41:05.059829+0000","last_clean":"2023-10-06T01:41:05.059829+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:41:05.059829+0000","last_undegraded":"2023-10-06T01:41:05.059829+0000","last_fullsized":"2023-10-06T01:41:05.059829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074919+0000","last_change":"2023-10-06T01:41:05.072432+0000","last_active":"2023-10-06T01:41:05.074919+0000","last_peered":"2023-10-06T01:41:05.074919+0000","last_clean":"2023-10-06T01:41:05.074919+0000","last_became_active":"2023-10-06T01:41:05.072154+0000","last_became_peered":"2023-10-06T01:41:05.072154+0000","last_unstale":"2023-10-06T01:41:05.074919+0000","last_undegraded":"2023-10-06T01:41:05.074919+0000","last_fullsized":"2023-10-06T01:41:05.074919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:55.047887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063306+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:41:05.063306+0000","last_peered":"2023-10-06T01:41:05.063306+0000","last_clean":"2023-10-06T01:41:05.063306+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:41:05.063306+0000","last_undegraded":"2023-10-06T01:41:05.063306+0000","last_fullsized":"2023-10-06T01:41:05.063306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068615+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:41:05.068615+0000","last_peered":"2023-10-06T01:41:05.068615+0000","last_clean":"2023-10-06T01:41:05.068615+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:41:05.068615+0000","last_undegraded":"2023-10-06T01:41:05.068615+0000","last_fullsized":"2023-10-06T01:41:05.068615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077358+0000","last_change":"2023-10-06T01:41:05.075392+0000","last_active":"2023-10-06T01:41:05.077358+0000","last_peered":"2023-10-06T01:41:05.077358+0000","last_clean":"2023-10-06T01:41:05.077358+0000","last_became_active":"2023-10-06T01:41:05.075108+0000","last_became_peered":"2023-10-06T01:41:05.075108+0000","last_unstale":"2023-10-06T01:41:05.077358+0000","last_undegraded":"2023-10-06T01:41:05.077358+0000","last_fullsized":"2023-10-06T01:41:05.077358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:43.179045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061850+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:41:05.061850+0000","last_peered":"2023-10-06T01:41:05.061850+0000","last_clean":"2023-10-06T01:41:05.061850+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:41:05.061850+0000","last_undegraded":"2023-10-06T01:41:05.061850+0000","last_fullsized":"2023-10-06T01:41:05.061850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063096+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:41:05.063096+0000","last_peered":"2023-10-06T01:41:05.063096+0000","last_clean":"2023-10-06T01:41:05.063096+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:41:05.063096+0000","last_undegraded":"2023-10-06T01:41:05.063096+0000","last_fullsized":"2023-10-06T01:41:05.063096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075586+0000","last_change":"2023-10-06T01:41:05.072321+0000","last_active":"2023-10-06T01:41:05.075586+0000","last_peered":"2023-10-06T01:41:05.075586+0000","last_clean":"2023-10-06T01:41:05.075586+0000","last_became_active":"2023-10-06T01:41:05.072037+0000","last_became_peered":"2023-10-06T01:41:05.072037+0000","last_unstale":"2023-10-06T01:41:05.075586+0000","last_undegraded":"2023-10-06T01:41:05.075586+0000","last_fullsized":"2023-10-06T01:41:05.075586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067494+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:41:05.067494+0000","last_peered":"2023-10-06T01:41:05.067494+0000","last_clean":"2023-10-06T01:41:05.067494+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:41:05.067494+0000","last_undegraded":"2023-10-06T01:41:05.067494+0000","last_fullsized":"2023-10-06T01:41:05.067494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062861+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:41:05.062861+0000","last_peered":"2023-10-06T01:41:05.062861+0000","last_clean":"2023-10-06T01:41:05.062861+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:41:05.062861+0000","last_undegraded":"2023-10-06T01:41:05.062861+0000","last_fullsized":"2023-10-06T01:41:05.062861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071654+0000","last_change":"2023-10-06T01:41:05.070017+0000","last_active":"2023-10-06T01:41:05.071654+0000","last_peered":"2023-10-06T01:41:05.071654+0000","last_clean":"2023-10-06T01:41:05.071654+0000","last_became_active":"2023-10-06T01:41:05.069741+0000","last_became_peered":"2023-10-06T01:41:05.069741+0000","last_unstale":"2023-10-06T01:41:05.071654+0000","last_undegraded":"2023-10-06T01:41:05.071654+0000","last_fullsized":"2023-10-06T01:41:05.071654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:36.570717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062619+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:41:05.062619+0000","last_peered":"2023-10-06T01:41:05.062619+0000","last_clean":"2023-10-06T01:41:05.062619+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:41:05.062619+0000","last_undegraded":"2023-10-06T01:41:05.062619+0000","last_fullsized":"2023-10-06T01:41:05.062619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067659+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:41:05.067659+0000","last_peered":"2023-10-06T01:41:05.067659+0000","last_clean":"2023-10-06T01:41:05.067659+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:41:05.067659+0000","last_undegraded":"2023-10-06T01:41:05.067659+0000","last_fullsized":"2023-10-06T01:41:05.067659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077336+0000","last_change":"2023-10-06T01:41:05.074701+0000","last_active":"2023-10-06T01:41:05.077336+0000","last_peered":"2023-10-06T01:41:05.077336+0000","last_clean":"2023-10-06T01:41:05.077336+0000","last_became_active":"2023-10-06T01:41:05.074479+0000","last_became_peered":"2023-10-06T01:41:05.074479+0000","last_unstale":"2023-10-06T01:41:05.077336+0000","last_undegraded":"2023-10-06T01:41:05.077336+0000","last_fullsized":"2023-10-06T01:41:05.077336+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:42.689091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063152+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:41:05.063152+0000","last_peered":"2023-10-06T01:41:05.063152+0000","last_clean":"2023-10-06T01:41:05.063152+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:41:05.063152+0000","last_undegraded":"2023-10-06T01:41:05.063152+0000","last_fullsized":"2023-10-06T01:41:05.063152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067927+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:41:05.067927+0000","last_peered":"2023-10-06T01:41:05.067927+0000","last_clean":"2023-10-06T01:41:05.067927+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:41:05.067927+0000","last_undegraded":"2023-10-06T01:41:05.067927+0000","last_fullsized":"2023-10-06T01:41:05.067927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071094+0000","last_change":"2023-10-06T01:41:05.068576+0000","last_active":"2023-10-06T01:41:05.071094+0000","last_peered":"2023-10-06T01:41:05.071094+0000","last_clean":"2023-10-06T01:41:05.071094+0000","last_became_active":"2023-10-06T01:41:05.068329+0000","last_became_peered":"2023-10-06T01:41:05.068329+0000","last_unstale":"2023-10-06T01:41:05.071094+0000","last_undegraded":"2023-10-06T01:41:05.071094+0000","last_fullsized":"2023-10-06T01:41:05.071094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063824+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:41:05.063824+0000","last_peered":"2023-10-06T01:41:05.063824+0000","last_clean":"2023-10-06T01:41:05.063824+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:41:05.063824+0000","last_undegraded":"2023-10-06T01:41:05.063824+0000","last_fullsized":"2023-10-06T01:41:05.063824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063903+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:41:05.063903+0000","last_peered":"2023-10-06T01:41:05.063903+0000","last_clean":"2023-10-06T01:41:05.063903+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:41:05.063903+0000","last_undegraded":"2023-10-06T01:41:05.063903+0000","last_fullsized":"2023-10-06T01:41:05.063903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.076999+0000","last_change":"2023-10-06T01:41:05.072828+0000","last_active":"2023-10-06T01:41:05.076999+0000","last_peered":"2023-10-06T01:41:05.076999+0000","last_clean":"2023-10-06T01:41:05.076999+0000","last_became_active":"2023-10-06T01:41:05.072570+0000","last_became_peered":"2023-10-06T01:41:05.072570+0000","last_unstale":"2023-10-06T01:41:05.076999+0000","last_undegraded":"2023-10-06T01:41:05.076999+0000","last_fullsized":"2023-10-06T01:41:05.076999+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:05.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064241+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:41:05.064241+0000","last_peered":"2023-10-06T01:41:05.064241+0000","last_clean":"2023-10-06T01:41:05.064241+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:41:05.064241+0000","last_undegraded":"2023-10-06T01:41:05.064241+0000","last_fullsized":"2023-10-06T01:41:05.064241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066774+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:41:05.066774+0000","last_peered":"2023-10-06T01:41:05.066774+0000","last_clean":"2023-10-06T01:41:05.066774+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:41:05.066774+0000","last_undegraded":"2023-10-06T01:41:05.066774+0000","last_fullsized":"2023-10-06T01:41:05.066774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077743+0000","last_change":"2023-10-06T01:41:05.075317+0000","last_active":"2023-10-06T01:41:05.077743+0000","last_peered":"2023-10-06T01:41:05.077743+0000","last_clean":"2023-10-06T01:41:05.077743+0000","last_became_active":"2023-10-06T01:41:05.075139+0000","last_became_peered":"2023-10-06T01:41:05.075139+0000","last_unstale":"2023-10-06T01:41:05.077743+0000","last_undegraded":"2023-10-06T01:41:05.077743+0000","last_fullsized":"2023-10-06T01:41:05.077743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062926+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:41:05.062926+0000","last_peered":"2023-10-06T01:41:05.062926+0000","last_clean":"2023-10-06T01:41:05.062926+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:41:05.062926+0000","last_undegraded":"2023-10-06T01:41:05.062926+0000","last_fullsized":"2023-10-06T01:41:05.062926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064314+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:41:05.064314+0000","last_peered":"2023-10-06T01:41:05.064314+0000","last_clean":"2023-10-06T01:41:05.064314+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:41:05.064314+0000","last_undegraded":"2023-10-06T01:41:05.064314+0000","last_fullsized":"2023-10-06T01:41:05.064314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074604+0000","last_change":"2023-10-06T01:41:05.069137+0000","last_active":"2023-10-06T01:41:05.074604+0000","last_peered":"2023-10-06T01:41:05.074604+0000","last_clean":"2023-10-06T01:41:05.074604+0000","last_became_active":"2023-10-06T01:41:05.068861+0000","last_became_peered":"2023-10-06T01:41:05.068861+0000","last_unstale":"2023-10-06T01:41:05.074604+0000","last_undegraded":"2023-10-06T01:41:05.074604+0000","last_fullsized":"2023-10-06T01:41:05.074604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:54.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063296+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:41:05.063296+0000","last_peered":"2023-10-06T01:41:05.063296+0000","last_clean":"2023-10-06T01:41:05.063296+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:41:05.063296+0000","last_undegraded":"2023-10-06T01:41:05.063296+0000","last_fullsized":"2023-10-06T01:41:05.063296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063779+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:41:05.063779+0000","last_peered":"2023-10-06T01:41:05.063779+0000","last_clean":"2023-10-06T01:41:05.063779+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:41:05.063779+0000","last_undegraded":"2023-10-06T01:41:05.063779+0000","last_fullsized":"2023-10-06T01:41:05.063779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073651+0000","last_change":"2023-10-06T01:41:05.071706+0000","last_active":"2023-10-06T01:41:05.073651+0000","last_peered":"2023-10-06T01:41:05.073651+0000","last_clean":"2023-10-06T01:41:05.073651+0000","last_became_active":"2023-10-06T01:41:05.071432+0000","last_became_peered":"2023-10-06T01:41:05.071432+0000","last_unstale":"2023-10-06T01:41:05.073651+0000","last_undegraded":"2023-10-06T01:41:05.073651+0000","last_fullsized":"2023-10-06T01:41:05.073651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070266+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:41:05.070266+0000","last_peered":"2023-10-06T01:41:05.070266+0000","last_clean":"2023-10-06T01:41:05.070266+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:41:05.070266+0000","last_undegraded":"2023-10-06T01:41:05.070266+0000","last_fullsized":"2023-10-06T01:41:05.070266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067322+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:41:05.067322+0000","last_peered":"2023-10-06T01:41:05.067322+0000","last_clean":"2023-10-06T01:41:05.067322+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:41:05.067322+0000","last_undegraded":"2023-10-06T01:41:05.067322+0000","last_fullsized":"2023-10-06T01:41:05.067322+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070176+0000","last_change":"2023-10-06T01:41:05.066490+0000","last_active":"2023-10-06T01:41:05.070176+0000","last_peered":"2023-10-06T01:41:05.070176+0000","last_clean":"2023-10-06T01:41:05.070176+0000","last_became_active":"2023-10-06T01:41:05.065985+0000","last_became_peered":"2023-10-06T01:41:05.065985+0000","last_unstale":"2023-10-06T01:41:05.070176+0000","last_undegraded":"2023-10-06T01:41:05.070176+0000","last_fullsized":"2023-10-06T01:41:05.070176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067405+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:41:05.067405+0000","last_peered":"2023-10-06T01:41:05.067405+0000","last_clean":"2023-10-06T01:41:05.067405+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:41:05.067405+0000","last_undegraded":"2023-10-06T01:41:05.067405+0000","last_fullsized":"2023-10-06T01:41:05.067405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069275+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:41:05.069275+0000","last_peered":"2023-10-06T01:41:05.069275+0000","last_clean":"2023-10-06T01:41:05.069275+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:41:05.069275+0000","last_undegraded":"2023-10-06T01:41:05.069275+0000","last_fullsized":"2023-10-06T01:41:05.069275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074516+0000","last_change":"2023-10-06T01:41:05.071088+0000","last_active":"2023-10-06T01:41:05.074516+0000","last_peered":"2023-10-06T01:41:05.074516+0000","last_clean":"2023-10-06T01:41:05.074516+0000","last_became_active":"2023-10-06T01:41:05.070727+0000","last_became_peered":"2023-10-06T01:41:05.070727+0000","last_unstale":"2023-10-06T01:41:05.074516+0000","last_undegraded":"2023-10-06T01:41:05.074516+0000","last_fullsized":"2023-10-06T01:41:05.074516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068853+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:41:05.068853+0000","last_peered":"2023-10-06T01:41:05.068853+0000","last_clean":"2023-10-06T01:41:05.068853+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:41:05.068853+0000","last_undegraded":"2023-10-06T01:41:05.068853+0000","last_fullsized":"2023-10-06T01:41:05.068853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'22","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060056+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:41:05.060056+0000","last_peered":"2023-10-06T01:41:05.060056+0000","last_clean":"2023-10-06T01:41:05.060056+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:41:05.060056+0000","last_undegraded":"2023-10-06T01:41:05.060056+0000","last_fullsized":"2023-10-06T01:41:05.060056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077429+0000","last_change":"2023-10-06T01:41:05.072554+0000","last_active":"2023-10-06T01:41:05.077429+0000","last_peered":"2023-10-06T01:41:05.077429+0000","last_clean":"2023-10-06T01:41:05.077429+0000","last_became_active":"2023-10-06T01:41:05.072311+0000","last_became_peered":"2023-10-06T01:41:05.072311+0000","last_unstale":"2023-10-06T01:41:05.077429+0000","last_undegraded":"2023-10-06T01:41:05.077429+0000","last_fullsized":"2023-10-06T01:41:05.077429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062171+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:41:05.062171+0000","last_peered":"2023-10-06T01:41:05.062171+0000","last_clean":"2023-10-06T01:41:05.062171+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:41:05.062171+0000","last_undegraded":"2023-10-06T01:41:05.062171+0000","last_fullsized":"2023-10-06T01:41:05.062171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067767+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:41:05.067767+0000","last_peered":"2023-10-06T01:41:05.067767+0000","last_clean":"2023-10-06T01:41:05.067767+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:41:05.067767+0000","last_undegraded":"2023-10-06T01:41:05.067767+0000","last_fullsized":"2023-10-06T01:41:05.067767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074931+0000","last_change":"2023-10-06T01:41:05.072492+0000","last_active":"2023-10-06T01:41:05.074931+0000","last_peered":"2023-10-06T01:41:05.074931+0000","last_clean":"2023-10-06T01:41:05.074931+0000","last_became_active":"2023-10-06T01:41:05.072225+0000","last_became_peered":"2023-10-06T01:41:05.072225+0000","last_unstale":"2023-10-06T01:41:05.074931+0000","last_undegraded":"2023-10-06T01:41:05.074931+0000","last_fullsized":"2023-10-06T01:41:05.074931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.539127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063929+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:41:05.063929+0000","last_peered":"2023-10-06T01:41:05.063929+0000","last_clean":"2023-10-06T01:41:05.063929+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:41:05.063929+0000","last_undegraded":"2023-10-06T01:41:05.063929+0000","last_fullsized":"2023-10-06T01:41:05.063929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067895+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:41:05.067895+0000","last_peered":"2023-10-06T01:41:05.067895+0000","last_clean":"2023-10-06T01:41:05.067895+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:41:05.067895+0000","last_undegraded":"2023-10-06T01:41:05.067895+0000","last_fullsized":"2023-10-06T01:41:05.067895+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077494+0000","last_change":"2023-10-06T01:41:05.072597+0000","last_active":"2023-10-06T01:41:05.077494+0000","last_peered":"2023-10-06T01:41:05.077494+0000","last_clean":"2023-10-06T01:41:05.077494+0000","last_became_active":"2023-10-06T01:41:05.072379+0000","last_became_peered":"2023-10-06T01:41:05.072379+0000","last_unstale":"2023-10-06T01:41:05.077494+0000","last_undegraded":"2023-10-06T01:41:05.077494+0000","last_fullsized":"2023-10-06T01:41:05.077494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067386+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:41:05.067386+0000","last_peered":"2023-10-06T01:41:05.067386+0000","last_clean":"2023-10-06T01:41:05.067386+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:41:05.067386+0000","last_undegraded":"2023-10-06T01:41:05.067386+0000","last_fullsized":"2023-10-06T01:41:05.067386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066926+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:41:05.066926+0000","last_peered":"2023-10-06T01:41:05.066926+0000","last_clean":"2023-10-06T01:41:05.066926+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:41:05.066926+0000","last_undegraded":"2023-10-06T01:41:05.066926+0000","last_fullsized":"2023-10-06T01:41:05.066926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073491+0000","last_change":"2023-10-06T01:41:05.070833+0000","last_active":"2023-10-06T01:41:05.073491+0000","last_peered":"2023-10-06T01:41:05.073491+0000","last_clean":"2023-10-06T01:41:05.073491+0000","last_became_active":"2023-10-06T01:41:05.070486+0000","last_became_peered":"2023-10-06T01:41:05.070486+0000","last_unstale":"2023-10-06T01:41:05.073491+0000","last_undegraded":"2023-10-06T01:41:05.073491+0000","last_fullsized":"2023-10-06T01:41:05.073491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068137+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:41:05.068137+0000","last_peered":"2023-10-06T01:41:05.068137+0000","last_clean":"2023-10-06T01:41:05.068137+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:41:05.068137+0000","last_undegraded":"2023-10-06T01:41:05.068137+0000","last_fullsized":"2023-10-06T01:41:05.068137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064395+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:41:05.064395+0000","last_peered":"2023-10-06T01:41:05.064395+0000","last_clean":"2023-10-06T01:41:05.064395+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:41:05.064395+0000","last_undegraded":"2023-10-06T01:41:05.064395+0000","last_fullsized":"2023-10-06T01:41:05.064395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071482+0000","last_change":"2023-10-06T01:41:05.069668+0000","last_active":"2023-10-06T01:41:05.071482+0000","last_peered":"2023-10-06T01:41:05.071482+0000","last_clean":"2023-10-06T01:41:05.071482+0000","last_became_active":"2023-10-06T01:41:05.069033+0000","last_became_peered":"2023-10-06T01:41:05.069033+0000","last_unstale":"2023-10-06T01:41:05.071482+0000","last_undegraded":"2023-10-06T01:41:05.071482+0000","last_fullsized":"2023-10-06T01:41:05.071482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:14:32.941476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059936+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:41:05.059936+0000","last_peered":"2023-10-06T01:41:05.059936+0000","last_clean":"2023-10-06T01:41:05.059936+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:41:05.059936+0000","last_undegraded":"2023-10-06T01:41:05.059936+0000","last_fullsized":"2023-10-06T01:41:05.059936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060008+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:41:05.060008+0000","last_peered":"2023-10-06T01:41:05.060008+0000","last_clean":"2023-10-06T01:41:05.060008+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:41:05.060008+0000","last_undegraded":"2023-10-06T01:41:05.060008+0000","last_fullsized":"2023-10-06T01:41:05.060008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075650+0000","last_change":"2023-10-06T01:41:05.072868+0000","last_active":"2023-10-06T01:41:05.075650+0000","last_peered":"2023-10-06T01:41:05.075650+0000","last_clean":"2023-10-06T01:41:05.075650+0000","last_became_active":"2023-10-06T01:41:05.072621+0000","last_became_peered":"2023-10-06T01:41:05.072621+0000","last_unstale":"2023-10-06T01:41:05.075650+0000","last_undegraded":"2023-10-06T01:41:05.075650+0000","last_fullsized":"2023-10-06T01:41:05.075650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067784+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:41:05.067784+0000","last_peered":"2023-10-06T01:41:05.067784+0000","last_clean":"2023-10-06T01:41:05.067784+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:41:05.067784+0000","last_undegraded":"2023-10-06T01:41:05.067784+0000","last_fullsized":"2023-10-06T01:41:05.067784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'3","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063389+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:41:05.063389+0000","last_peered":"2023-10-06T01:41:05.063389+0000","last_clean":"2023-10-06T01:41:05.063389+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:41:05.063389+0000","last_undegraded":"2023-10-06T01:41:05.063389+0000","last_fullsized":"2023-10-06T01:41:05.063389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.080676+0000","last_change":"2023-10-06T01:41:05.073304+0000","last_active":"2023-10-06T01:41:05.080676+0000","last_peered":"2023-10-06T01:41:05.080676+0000","last_clean":"2023-10-06T01:41:05.080676+0000","last_became_active":"2023-10-06T01:41:05.073056+0000","last_became_peered":"2023-10-06T01:41:05.073056+0000","last_unstale":"2023-10-06T01:41:05.080676+0000","last_undegraded":"2023-10-06T01:41:05.080676+0000","last_fullsized":"2023-10-06T01:41:05.080676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:14.536410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061740+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:41:05.061740+0000","last_peered":"2023-10-06T01:41:05.061740+0000","last_clean":"2023-10-06T01:41:05.061740+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:41:05.061740+0000","last_undegraded":"2023-10-06T01:41:05.061740+0000","last_fullsized":"2023-10-06T01:41:05.061740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'2","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067860+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:41:05.067860+0000","last_peered":"2023-10-06T01:41:05.067860+0000","last_clean":"2023-10-06T01:41:05.067860+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:41:05.067860+0000","last_undegraded":"2023-10-06T01:41:05.067860+0000","last_fullsized":"2023-10-06T01:41:05.067860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068813+0000","last_change":"2023-10-06T01:41:05.066269+0000","last_active":"2023-10-06T01:41:05.068813+0000","last_peered":"2023-10-06T01:41:05.068813+0000","last_clean":"2023-10-06T01:41:05.068813+0000","last_became_active":"2023-10-06T01:41:05.065979+0000","last_became_peered":"2023-10-06T01:41:05.065979+0000","last_unstale":"2023-10-06T01:41:05.068813+0000","last_undegraded":"2023-10-06T01:41:05.068813+0000","last_fullsized":"2023-10-06T01:41:05.068813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062811+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:41:05.062811+0000","last_peered":"2023-10-06T01:41:05.062811+0000","last_clean":"2023-10-06T01:41:05.062811+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:41:05.062811+0000","last_undegraded":"2023-10-06T01:41:05.062811+0000","last_fullsized":"2023-10-06T01:41:05.062811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067544+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:41:05.067544+0000","last_peered":"2023-10-06T01:41:05.067544+0000","last_clean":"2023-10-06T01:41:05.067544+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:41:05.067544+0000","last_undegraded":"2023-10-06T01:41:05.067544+0000","last_fullsized":"2023-10-06T01:41:05.067544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070058+0000","last_change":"2023-10-06T01:41:05.064865+0000","last_active":"2023-10-06T01:41:05.070058+0000","last_peered":"2023-10-06T01:41:05.070058+0000","last_clean":"2023-10-06T01:41:05.070058+0000","last_became_active":"2023-10-06T01:41:05.064644+0000","last_became_peered":"2023-10-06T01:41:05.064644+0000","last_unstale":"2023-10-06T01:41:05.070058+0000","last_undegraded":"2023-10-06T01:41:05.070058+0000","last_fullsized":"2023-10-06T01:41:05.070058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:27.127112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067040+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:41:05.067040+0000","last_peered":"2023-10-06T01:41:05.067040+0000","last_clean":"2023-10-06T01:41:05.067040+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:41:05.067040+0000","last_undegraded":"2023-10-06T01:41:05.067040+0000","last_fullsized":"2023-10-06T01:41:05.067040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'4","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067848+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:41:05.067848+0000","last_peered":"2023-10-06T01:41:05.067848+0000","last_clean":"2023-10-06T01:41:05.067848+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:41:05.067848+0000","last_undegraded":"2023-10-06T01:41:05.067848+0000","last_fullsized":"2023-10-06T01:41:05.067848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074693+0000","last_change":"2023-10-06T01:41:05.069995+0000","last_active":"2023-10-06T01:41:05.074693+0000","last_peered":"2023-10-06T01:41:05.074693+0000","last_clean":"2023-10-06T01:41:05.074693+0000","last_became_active":"2023-10-06T01:41:05.069709+0000","last_became_peered":"2023-10-06T01:41:05.069709+0000","last_unstale":"2023-10-06T01:41:05.074693+0000","last_undegraded":"2023-10-06T01:41:05.074693+0000","last_fullsized":"2023-10-06T01:41:05.074693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:50.555907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062679+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:41:05.062679+0000","last_peered":"2023-10-06T01:41:05.062679+0000","last_clean":"2023-10-06T01:41:05.062679+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:41:05.062679+0000","last_undegraded":"2023-10-06T01:41:05.062679+0000","last_fullsized":"2023-10-06T01:41:05.062679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066603+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:41:05.066603+0000","last_peered":"2023-10-06T01:41:05.066603+0000","last_clean":"2023-10-06T01:41:05.066603+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:41:05.066603+0000","last_undegraded":"2023-10-06T01:41:05.066603+0000","last_fullsized":"2023-10-06T01:41:05.066603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077363+0000","last_change":"2023-10-06T01:41:05.071858+0000","last_active":"2023-10-06T01:41:05.077363+0000","last_peered":"2023-10-06T01:41:05.077363+0000","last_clean":"2023-10-06T01:41:05.077363+0000","last_became_active":"2023-10-06T01:41:05.071590+0000","last_became_peered":"2023-10-06T01:41:05.071590+0000","last_unstale":"2023-10-06T01:41:05.077363+0000","last_undegraded":"2023-10-06T01:41:05.077363+0000","last_fullsized":"2023-10-06T01:41:05.077363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:39.551110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059831+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:41:05.059831+0000","last_peered":"2023-10-06T01:41:05.059831+0000","last_clean":"2023-10-06T01:41:05.059831+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:41:05.059831+0000","last_undegraded":"2023-10-06T01:41:05.059831+0000","last_fullsized":"2023-10-06T01:41:05.059831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063540+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:41:05.063540+0000","last_peered":"2023-10-06T01:41:05.063540+0000","last_clean":"2023-10-06T01:41:05.063540+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:41:05.063540+0000","last_undegraded":"2023-10-06T01:41:05.063540+0000","last_fullsized":"2023-10-06T01:41:05.063540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069555+0000","last_change":"2023-10-06T01:41:05.068429+0000","last_active":"2023-10-06T01:41:05.069555+0000","last_peered":"2023-10-06T01:41:05.069555+0000","last_clean":"2023-10-06T01:41:05.069555+0000","last_became_active":"2023-10-06T01:41:05.068180+0000","last_became_peered":"2023-10-06T01:41:05.068180+0000","last_unstale":"2023-10-06T01:41:05.069555+0000","last_undegraded":"2023-10-06T01:41:05.069555+0000","last_fullsized":"2023-10-06T01:41:05.069555+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"36'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063653+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:41:05.063653+0000","last_peered":"2023-10-06T01:41:05.063653+0000","last_clean":"2023-10-06T01:41:05.063653+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:41:05.063653+0000","last_undegraded":"2023-10-06T01:41:05.063653+0000","last_fullsized":"2023-10-06T01:41:05.063653+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067505+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:41:05.067505+0000","last_peered":"2023-10-06T01:41:05.067505+0000","last_clean":"2023-10-06T01:41:05.067505+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:41:05.067505+0000","last_undegraded":"2023-10-06T01:41:05.067505+0000","last_fullsized":"2023-10-06T01:41:05.067505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077459+0000","last_change":"2023-10-06T01:41:05.071544+0000","last_active":"2023-10-06T01:41:05.077459+0000","last_peered":"2023-10-06T01:41:05.077459+0000","last_clean":"2023-10-06T01:41:05.077459+0000","last_became_active":"2023-10-06T01:41:05.071300+0000","last_became_peered":"2023-10-06T01:41:05.071300+0000","last_unstale":"2023-10-06T01:41:05.077459+0000","last_undegraded":"2023-10-06T01:41:05.077459+0000","last_fullsized":"2023-10-06T01:41:05.077459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067681+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:41:05.067681+0000","last_peered":"2023-10-06T01:41:05.067681+0000","last_clean":"2023-10-06T01:41:05.067681+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:41:05.067681+0000","last_undegraded":"2023-10-06T01:41:05.067681+0000","last_fullsized":"2023-10-06T01:41:05.067681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063996+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:41:05.063996+0000","last_peered":"2023-10-06T01:41:05.063996+0000","last_clean":"2023-10-06T01:41:05.063996+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:41:05.063996+0000","last_undegraded":"2023-10-06T01:41:05.063996+0000","last_fullsized":"2023-10-06T01:41:05.063996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062842+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:05.062842+0000","last_peered":"2023-10-06T01:41:05.062842+0000","last_clean":"2023-10-06T01:41:05.062842+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:05.062842+0000","last_undegraded":"2023-10-06T01:41:05.062842+0000","last_fullsized":"2023-10-06T01:41:05.062842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074573+0000","last_change":"2023-10-06T01:41:05.070827+0000","last_active":"2023-10-06T01:41:05.074573+0000","last_peered":"2023-10-06T01:41:05.074573+0000","last_clean":"2023-10-06T01:41:05.074573+0000","last_became_active":"2023-10-06T01:41:05.070649+0000","last_became_peered":"2023-10-06T01:41:05.070649+0000","last_unstale":"2023-10-06T01:41:05.074573+0000","last_undegraded":"2023-10-06T01:41:05.074573+0000","last_fullsized":"2023-10-06T01:41:05.074573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:26.468333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067196+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:41:05.067196+0000","last_peered":"2023-10-06T01:41:05.067196+0000","last_clean":"2023-10-06T01:41:05.067196+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:41:05.067196+0000","last_undegraded":"2023-10-06T01:41:05.067196+0000","last_fullsized":"2023-10-06T01:41:05.067196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067890+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:41:05.067890+0000","last_peered":"2023-10-06T01:41:05.067890+0000","last_clean":"2023-10-06T01:41:05.067890+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:41:05.067890+0000","last_undegraded":"2023-10-06T01:41:05.067890+0000","last_fullsized":"2023-10-06T01:41:05.067890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067611+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:05.067611+0000","last_peered":"2023-10-06T01:41:05.067611+0000","last_clean":"2023-10-06T01:41:05.067611+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:05.067611+0000","last_undegraded":"2023-10-06T01:41:05.067611+0000","last_fullsized":"2023-10-06T01:41:05.067611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071384+0000","last_change":"2023-10-06T01:41:05.068635+0000","last_active":"2023-10-06T01:41:05.071384+0000","last_peered":"2023-10-06T01:41:05.071384+0000","last_clean":"2023-10-06T01:41:05.071384+0000","last_became_active":"2023-10-06T01:41:05.068364+0000","last_became_peered":"2023-10-06T01:41:05.068364+0000","last_unstale":"2023-10-06T01:41:05.071384+0000","last_undegraded":"2023-10-06T01:41:05.071384+0000","last_fullsized":"2023-10-06T01:41:05.071384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:53.027266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069163+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:41:05.069163+0000","last_peered":"2023-10-06T01:41:05.069163+0000","last_clean":"2023-10-06T01:41:05.069163+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:41:05.069163+0000","last_undegraded":"2023-10-06T01:41:05.069163+0000","last_fullsized":"2023-10-06T01:41:05.069163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062581+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:41:05.062581+0000","last_peered":"2023-10-06T01:41:05.062581+0000","last_clean":"2023-10-06T01:41:05.062581+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:41:05.062581+0000","last_undegraded":"2023-10-06T01:41:05.062581+0000","last_fullsized":"2023-10-06T01:41:05.062581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067978+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:05.067978+0000","last_peered":"2023-10-06T01:41:05.067978+0000","last_clean":"2023-10-06T01:41:05.067978+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:05.067978+0000","last_undegraded":"2023-10-06T01:41:05.067978+0000","last_fullsized":"2023-10-06T01:41:05.067978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075630+0000","last_change":"2023-10-06T01:41:05.072823+0000","last_active":"2023-10-06T01:41:05.075630+0000","last_peered":"2023-10-06T01:41:05.075630+0000","last_clean":"2023-10-06T01:41:05.075630+0000","last_became_active":"2023-10-06T01:41:05.072511+0000","last_became_peered":"2023-10-06T01:41:05.072511+0000","last_unstale":"2023-10-06T01:41:05.075630+0000","last_undegraded":"2023-10-06T01:41:05.075630+0000","last_fullsized":"2023-10-06T01:41:05.075630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064300+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:41:05.064300+0000","last_peered":"2023-10-06T01:41:05.064300+0000","last_clean":"2023-10-06T01:41:05.064300+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:41:05.064300+0000","last_undegraded":"2023-10-06T01:41:05.064300+0000","last_fullsized":"2023-10-06T01:41:05.064300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069409+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:41:05.069409+0000","last_peered":"2023-10-06T01:41:05.069409+0000","last_clean":"2023-10-06T01:41:05.069409+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:41:05.069409+0000","last_undegraded":"2023-10-06T01:41:05.069409+0000","last_fullsized":"2023-10-06T01:41:05.069409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067997+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:05.067997+0000","last_peered":"2023-10-06T01:41:05.067997+0000","last_clean":"2023-10-06T01:41:05.067997+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:05.067997+0000","last_undegraded":"2023-10-06T01:41:05.067997+0000","last_fullsized":"2023-10-06T01:41:05.067997+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075181+0000","last_change":"2023-10-06T01:41:05.072739+0000","last_active":"2023-10-06T01:41:05.075181+0000","last_peered":"2023-10-06T01:41:05.075181+0000","last_clean":"2023-10-06T01:41:05.075181+0000","last_became_active":"2023-10-06T01:41:05.072486+0000","last_became_peered":"2023-10-06T01:41:05.072486+0000","last_unstale":"2023-10-06T01:41:05.075181+0000","last_undegraded":"2023-10-06T01:41:05.075181+0000","last_fullsized":"2023-10-06T01:41:05.075181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:44.012384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067237+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:41:05.067237+0000","last_peered":"2023-10-06T01:41:05.067237+0000","last_clean":"2023-10-06T01:41:05.067237+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:41:05.067237+0000","last_undegraded":"2023-10-06T01:41:05.067237+0000","last_fullsized":"2023-10-06T01:41:05.067237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068333+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:41:05.068333+0000","last_peered":"2023-10-06T01:41:05.068333+0000","last_clean":"2023-10-06T01:41:05.068333+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:41:05.068333+0000","last_undegraded":"2023-10-06T01:41:05.068333+0000","last_fullsized":"2023-10-06T01:41:05.068333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067888+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:05.067888+0000","last_peered":"2023-10-06T01:41:05.067888+0000","last_clean":"2023-10-06T01:41:05.067888+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:05.067888+0000","last_undegraded":"2023-10-06T01:41:05.067888+0000","last_fullsized":"2023-10-06T01:41:05.067888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074585+0000","last_change":"2023-10-06T01:41:05.071521+0000","last_active":"2023-10-06T01:41:05.074585+0000","last_peered":"2023-10-06T01:41:05.074585+0000","last_clean":"2023-10-06T01:41:05.074585+0000","last_became_active":"2023-10-06T01:41:05.071272+0000","last_became_peered":"2023-10-06T01:41:05.071272+0000","last_unstale":"2023-10-06T01:41:05.074585+0000","last_undegraded":"2023-10-06T01:41:05.074585+0000","last_fullsized":"2023-10-06T01:41:05.074585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.431329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'3","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068093+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:41:05.068093+0000","last_peered":"2023-10-06T01:41:05.068093+0000","last_clean":"2023-10-06T01:41:05.068093+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:41:05.068093+0000","last_undegraded":"2023-10-06T01:41:05.068093+0000","last_fullsized":"2023-10-06T01:41:05.068093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066856+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:05.066856+0000","last_peered":"2023-10-06T01:41:05.066856+0000","last_clean":"2023-10-06T01:41:05.066856+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:05.066856+0000","last_undegraded":"2023-10-06T01:41:05.066856+0000","last_fullsized":"2023-10-06T01:41:05.066856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077722+0000","last_change":"2023-10-06T01:41:05.073694+0000","last_active":"2023-10-06T01:41:05.077722+0000","last_peered":"2023-10-06T01:41:05.077722+0000","last_clean":"2023-10-06T01:41:05.077722+0000","last_became_active":"2023-10-06T01:41:05.073261+0000","last_became_peered":"2023-10-06T01:41:05.073261+0000","last_unstale":"2023-10-06T01:41:05.077722+0000","last_undegraded":"2023-10-06T01:41:05.077722+0000","last_fullsized":"2023-10-06T01:41:05.077722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066331+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:41:05.066331+0000","last_peered":"2023-10-06T01:41:05.066331+0000","last_clean":"2023-10-06T01:41:05.066331+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:41:05.066331+0000","last_undegraded":"2023-10-06T01:41:05.066331+0000","last_fullsized":"2023-10-06T01:41:05.066331+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062956+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:41:05.062956+0000","last_peered":"2023-10-06T01:41:05.062956+0000","last_clean":"2023-10-06T01:41:05.062956+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:41:05.062956+0000","last_undegraded":"2023-10-06T01:41:05.062956+0000","last_fullsized":"2023-10-06T01:41:05.062956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062529+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:05.062529+0000","last_peered":"2023-10-06T01:41:05.062529+0000","last_clean":"2023-10-06T01:41:05.062529+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:05.062529+0000","last_undegraded":"2023-10-06T01:41:05.062529+0000","last_fullsized":"2023-10-06T01:41:05.062529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.072041+0000","last_change":"2023-10-06T01:41:05.070184+0000","last_active":"2023-10-06T01:41:05.072041+0000","last_peered":"2023-10-06T01:41:05.072041+0000","last_clean":"2023-10-06T01:41:05.072041+0000","last_became_active":"2023-10-06T01:41:05.069937+0000","last_became_peered":"2023-10-06T01:41:05.069937+0000","last_unstale":"2023-10-06T01:41:05.072041+0000","last_undegraded":"2023-10-06T01:41:05.072041+0000","last_fullsized":"2023-10-06T01:41:05.072041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"36'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062666+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:41:05.062666+0000","last_peered":"2023-10-06T01:41:05.062666+0000","last_clean":"2023-10-06T01:41:05.062666+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:41:05.062666+0000","last_undegraded":"2023-10-06T01:41:05.062666+0000","last_fullsized":"2023-10-06T01:41:05.062666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064473+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:41:05.064473+0000","last_peered":"2023-10-06T01:41:05.064473+0000","last_clean":"2023-10-06T01:41:05.064473+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:41:05.064473+0000","last_undegraded":"2023-10-06T01:41:05.064473+0000","last_fullsized":"2023-10-06T01:41:05.064473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062127+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:05.062127+0000","last_peered":"2023-10-06T01:41:05.062127+0000","last_clean":"2023-10-06T01:41:05.062127+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:05.062127+0000","last_undegraded":"2023-10-06T01:41:05.062127+0000","last_fullsized":"2023-10-06T01:41:05.062127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.88},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.755},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.309}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.621}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":237748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.312}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":239101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41199999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.622},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:13.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:13.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.757+0000 7f813213e700 1 Processor -- start 2023-10-06T01:41:13.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.757+0000 7f813213e700 1 -- start start 2023-10-06T01:41:13.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f813213e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 0x7f812c0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f813213e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f812c0b49e0 0x7f812c0aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f813213e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0b0ca0 con 0x7f812c0b07a0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f813213e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c0b0f80 con 0x7f812c0b49e0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f813213e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b1260 con 0x7f812c1595a0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f812affd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f812c0b49e0 0x7f812c0aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f812affd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f812c0b49e0 0x7f812c0aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35766/0 (socket says 172.21.15.98:35766) 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.758+0000 7f812affd700 1 -- 172.21.15.98:0/3113942812 learned_addr learned my addr 172.21.15.98:0/3113942812 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 0x7f812c0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812affd700 1 -- 172.21.15.98:0/3113942812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 msgr2=0x7f812c0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812affd700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 0x7f812c0aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812affd700 1 -- 172.21.15.98:0/3113942812 >> v1:172.21.15.184:6789/0 conn(0x7f812c0b07a0 legacy=0x7f812c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812affd700 1 -- 172.21.15.98:0/3113942812 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0b1540 con 0x7f812c0b49e0 2023-10-06T01:41:13.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 0x7f812c0aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812affd700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c0b49e0 0x7f812c0aaab0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f81200099b0 tx=0x7f8120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9dcfc91cf33f43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812a7fc700 1 -- 172.21.15.98:0/3113942812 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000b820 con 0x7f812c0b49e0 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812a7fc700 1 -- 172.21.15.98:0/3113942812 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812000b980 con 0x7f812c0b49e0 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.759+0000 7f812a7fc700 1 -- 172.21.15.98:0/3113942812 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000bd20 con 0x7f812c0b49e0 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 -- 172.21.15.98:0/3113942812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c0b49e0 msgr2=0x7f812c0aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c0b49e0 0x7f812c0aaab0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f81200099b0 tx=0x7f8120009250 comp rx=0 tx=0).stop 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 -- 172.21.15.98:0/3113942812 shutdown_connections 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c1595a0 0x7f812c0aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 --2- 172.21.15.98:0/3113942812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c0b49e0 0x7f812c0aaab0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 -- 172.21.15.98:0/3113942812 >> 172.21.15.98:0/3113942812 conn(0x7f812c099830 msgr2=0x7f812c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 -- 172.21.15.98:0/3113942812 shutdown_connections 2023-10-06T01:41:13.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.760+0000 7f813213e700 1 -- 172.21.15.98:0/3113942812 wait complete. 2023-10-06T01:41:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.761+0000 7f813213e700 1 Processor -- start 2023-10-06T01:41:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.761+0000 7f813213e700 1 -- start start 2023-10-06T01:41:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 0x7f812c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 0x7f812c16d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c15f280 con 0x7f812c1595a0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c15c890 con 0x7f812c0b07a0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c09b750 con 0x7f812c0b49e0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35778/0 (socket says 172.21.15.98:35778) 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 -- 172.21.15.98:0/3058205592 learned_addr learned my addr 172.21.15.98:0/3058205592 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 0x7f812c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812affd700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 0x7f812c16d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 -- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 msgr2=0x7f812c16d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 0x7f812c16d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 -- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 msgr2=0x7f812c154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 0x7f812c154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 -- 172.21.15.98:0/3058205592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200094f0 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812affd700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 0x7f812c16d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 0x7f812c154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f812bfff700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f812400c960 tx=0x7f81240026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400cea0 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812400a7a0 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.762+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c15efa0 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.763+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400ab60 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.763+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c15c520 con 0x7f812c1595a0 2023-10-06T01:41:13.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.764+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c064ad0 con 0x7f812c1595a0 2023-10-06T01:41:13.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.764+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8124012420 con 0x7f812c1595a0 2023-10-06T01:41:13.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.764+0000 7f81137fe700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 0x7f81180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:13.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.764+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f8124084ec0 con 0x7f812c1595a0 2023-10-06T01:41:13.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.764+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 0x7f81180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:13.769 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.765+0000 7f812b7fe700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 0x7f81180693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f811c00ab10 tx=0x7f811c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:13.770 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.769+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8124051c70 con 0x7f812c1595a0 2023-10-06T01:41:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.948+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f812c15c6e0 con 0x7f8118066f30 2023-10-06T01:41:13.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.958+0000 7f81137fe700 1 -- 172.21.15.98:0/3058205592 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+392193 (secure 0 0 0) 0x7f812c15c6e0 con 0x7f8118066f30 2023-10-06T01:41:13.964 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 msgr2=0x7f81180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 0x7f81180693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f811c00ab10 tx=0x7f811c008040 comp rx=0 tx=0).stop 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 msgr2=0x7f812c16d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f812400c960 tx=0x7f81240026e0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 shutdown_connections 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8118066f30 0x7f81180693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c1595a0 0x7f812c16d690 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f812c0b07a0 0x7f812c154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.961+0000 7f813213e700 1 --2- 172.21.15.98:0/3058205592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f812c0b49e0 0x7f812c16d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.962+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 >> 172.21.15.98:0/3058205592 conn(0x7f812c099830 msgr2=0x7f812c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.962+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 shutdown_connections 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:13.962+0000 7f813213e700 1 -- 172.21.15.98:0/3058205592 wait complete. 2023-10-06T01:41:13.968 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:13.975 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-06T01:41:13.804215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395748,"kb_used_data":17444,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578972,"statfs":{"total":773094113280,"available":770640867328,"internally_reserved":0,"allocated":17862656,"data_stored":15675103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.745668"},"pg_stats":[{"pgid":"7.3f","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070953+0000","last_change":"2023-10-06T01:40:42.478358+0000","last_active":"2023-10-06T01:41:05.070953+0000","last_peered":"2023-10-06T01:41:05.070953+0000","last_clean":"2023-10-06T01:41:05.070953+0000","last_became_active":"2023-10-06T01:40:42.478086+0000","last_became_peered":"2023-10-06T01:40:42.478086+0000","last_unstale":"2023-10-06T01:41:05.070953+0000","last_undegraded":"2023-10-06T01:41:05.070953+0000","last_fullsized":"2023-10-06T01:41:05.070953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:39.751796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068055+0000","last_change":"2023-10-06T01:40:42.486453+0000","last_active":"2023-10-06T01:41:05.068055+0000","last_peered":"2023-10-06T01:41:05.068055+0000","last_clean":"2023-10-06T01:41:05.068055+0000","last_became_active":"2023-10-06T01:40:42.486181+0000","last_became_peered":"2023-10-06T01:40:42.486181+0000","last_unstale":"2023-10-06T01:41:05.068055+0000","last_undegraded":"2023-10-06T01:41:05.068055+0000","last_fullsized":"2023-10-06T01:41:05.068055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:57.116953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062731+0000","last_change":"2023-10-06T01:40:42.475410+0000","last_active":"2023-10-06T01:41:05.062731+0000","last_peered":"2023-10-06T01:41:05.062731+0000","last_clean":"2023-10-06T01:41:05.062731+0000","last_became_active":"2023-10-06T01:40:42.475103+0000","last_became_peered":"2023-10-06T01:40:42.475103+0000","last_unstale":"2023-10-06T01:41:05.062731+0000","last_undegraded":"2023-10-06T01:41:05.062731+0000","last_fullsized":"2023-10-06T01:41:05.062731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:49:38.993137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064200+0000","last_change":"2023-10-06T01:40:42.479027+0000","last_active":"2023-10-06T01:41:05.064200+0000","last_peered":"2023-10-06T01:41:05.064200+0000","last_clean":"2023-10-06T01:41:05.064200+0000","last_became_active":"2023-10-06T01:40:42.477665+0000","last_became_peered":"2023-10-06T01:40:42.477665+0000","last_unstale":"2023-10-06T01:41:05.064200+0000","last_undegraded":"2023-10-06T01:41:05.064200+0000","last_fullsized":"2023-10-06T01:41:05.064200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067188+0000","last_change":"2023-10-06T01:40:42.486344+0000","last_active":"2023-10-06T01:41:05.067188+0000","last_peered":"2023-10-06T01:41:05.067188+0000","last_clean":"2023-10-06T01:41:05.067188+0000","last_became_active":"2023-10-06T01:40:42.486024+0000","last_became_peered":"2023-10-06T01:40:42.486024+0000","last_unstale":"2023-10-06T01:41:05.067188+0000","last_undegraded":"2023-10-06T01:41:05.067188+0000","last_fullsized":"2023-10-06T01:41:05.067188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067168+0000","last_change":"2023-10-06T01:40:42.477586+0000","last_active":"2023-10-06T01:41:05.067168+0000","last_peered":"2023-10-06T01:41:05.067168+0000","last_clean":"2023-10-06T01:41:05.067168+0000","last_became_active":"2023-10-06T01:40:42.477345+0000","last_became_peered":"2023-10-06T01:40:42.477345+0000","last_unstale":"2023-10-06T01:41:05.067168+0000","last_undegraded":"2023-10-06T01:41:05.067168+0000","last_fullsized":"2023-10-06T01:41:05.067168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068070+0000","last_change":"2023-10-06T01:40:42.477100+0000","last_active":"2023-10-06T01:41:05.068070+0000","last_peered":"2023-10-06T01:41:05.068070+0000","last_clean":"2023-10-06T01:41:05.068070+0000","last_became_active":"2023-10-06T01:40:42.476845+0000","last_became_peered":"2023-10-06T01:40:42.476845+0000","last_unstale":"2023-10-06T01:41:05.068070+0000","last_undegraded":"2023-10-06T01:41:05.068070+0000","last_fullsized":"2023-10-06T01:41:05.068070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.023998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063265+0000","last_change":"2023-10-06T01:40:42.479538+0000","last_active":"2023-10-06T01:41:05.063265+0000","last_peered":"2023-10-06T01:41:05.063265+0000","last_clean":"2023-10-06T01:41:05.063265+0000","last_became_active":"2023-10-06T01:40:42.479348+0000","last_became_peered":"2023-10-06T01:40:42.479348+0000","last_unstale":"2023-10-06T01:41:05.063265+0000","last_undegraded":"2023-10-06T01:41:05.063265+0000","last_fullsized":"2023-10-06T01:41:05.063265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"36'11","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067676+0000","last_change":"2023-10-06T01:40:42.478626+0000","last_active":"2023-10-06T01:41:05.067676+0000","last_peered":"2023-10-06T01:41:05.067676+0000","last_clean":"2023-10-06T01:41:05.067676+0000","last_became_active":"2023-10-06T01:40:42.478367+0000","last_became_peered":"2023-10-06T01:40:42.478367+0000","last_unstale":"2023-10-06T01:41:05.067676+0000","last_undegraded":"2023-10-06T01:41:05.067676+0000","last_fullsized":"2023-10-06T01:41:05.067676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:05.225079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064867+0000","last_change":"2023-10-06T01:40:42.480200+0000","last_active":"2023-10-06T01:41:05.064867+0000","last_peered":"2023-10-06T01:41:05.064867+0000","last_clean":"2023-10-06T01:41:05.064867+0000","last_became_active":"2023-10-06T01:40:42.479744+0000","last_became_peered":"2023-10-06T01:40:42.479744+0000","last_unstale":"2023-10-06T01:41:05.064867+0000","last_undegraded":"2023-10-06T01:41:05.064867+0000","last_fullsized":"2023-10-06T01:41:05.064867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064219+0000","last_change":"2023-10-06T01:40:42.481292+0000","last_active":"2023-10-06T01:41:05.064219+0000","last_peered":"2023-10-06T01:41:05.064219+0000","last_clean":"2023-10-06T01:41:05.064219+0000","last_became_active":"2023-10-06T01:40:42.480031+0000","last_became_peered":"2023-10-06T01:40:42.480031+0000","last_unstale":"2023-10-06T01:41:05.064219+0000","last_undegraded":"2023-10-06T01:41:05.064219+0000","last_fullsized":"2023-10-06T01:41:05.064219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066867+0000","last_change":"2023-10-06T01:40:42.488464+0000","last_active":"2023-10-06T01:41:05.066867+0000","last_peered":"2023-10-06T01:41:05.066867+0000","last_clean":"2023-10-06T01:41:05.066867+0000","last_became_active":"2023-10-06T01:40:42.488211+0000","last_became_peered":"2023-10-06T01:40:42.488211+0000","last_unstale":"2023-10-06T01:41:05.066867+0000","last_undegraded":"2023-10-06T01:41:05.066867+0000","last_fullsized":"2023-10-06T01:41:05.066867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:34.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"36'10","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059981+0000","last_change":"2023-10-06T01:40:42.472916+0000","last_active":"2023-10-06T01:41:05.059981+0000","last_peered":"2023-10-06T01:41:05.059981+0000","last_clean":"2023-10-06T01:41:05.059981+0000","last_became_active":"2023-10-06T01:40:42.472618+0000","last_became_peered":"2023-10-06T01:40:42.472618+0000","last_unstale":"2023-10-06T01:41:05.059981+0000","last_undegraded":"2023-10-06T01:41:05.059981+0000","last_fullsized":"2023-10-06T01:41:05.059981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063679+0000","last_change":"2023-10-06T01:40:42.480979+0000","last_active":"2023-10-06T01:41:05.063679+0000","last_peered":"2023-10-06T01:41:05.063679+0000","last_clean":"2023-10-06T01:41:05.063679+0000","last_became_active":"2023-10-06T01:40:42.480702+0000","last_became_peered":"2023-10-06T01:40:42.480702+0000","last_unstale":"2023-10-06T01:41:05.063679+0000","last_undegraded":"2023-10-06T01:41:05.063679+0000","last_fullsized":"2023-10-06T01:41:05.063679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066663+0000","last_change":"2023-10-06T01:40:42.477751+0000","last_active":"2023-10-06T01:41:05.066663+0000","last_peered":"2023-10-06T01:41:05.066663+0000","last_clean":"2023-10-06T01:41:05.066663+0000","last_became_active":"2023-10-06T01:40:42.477080+0000","last_became_peered":"2023-10-06T01:40:42.477080+0000","last_unstale":"2023-10-06T01:41:05.066663+0000","last_undegraded":"2023-10-06T01:41:05.066663+0000","last_fullsized":"2023-10-06T01:41:05.066663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061713+0000","last_change":"2023-10-06T01:40:42.478135+0000","last_active":"2023-10-06T01:41:05.061713+0000","last_peered":"2023-10-06T01:41:05.061713+0000","last_clean":"2023-10-06T01:41:05.061713+0000","last_became_active":"2023-10-06T01:40:42.477885+0000","last_became_peered":"2023-10-06T01:40:42.477885+0000","last_unstale":"2023-10-06T01:41:05.061713+0000","last_undegraded":"2023-10-06T01:41:05.061713+0000","last_fullsized":"2023-10-06T01:41:05.061713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:39.225072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"36'9","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063591+0000","last_change":"2023-10-06T01:40:42.484255+0000","last_active":"2023-10-06T01:41:05.063591+0000","last_peered":"2023-10-06T01:41:05.063591+0000","last_clean":"2023-10-06T01:41:05.063591+0000","last_became_active":"2023-10-06T01:40:42.483957+0000","last_became_peered":"2023-10-06T01:40:42.483957+0000","last_unstale":"2023-10-06T01:41:05.063591+0000","last_undegraded":"2023-10-06T01:41:05.063591+0000","last_fullsized":"2023-10-06T01:41:05.063591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:13.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062553+0000","last_change":"2023-10-06T01:40:42.480446+0000","last_active":"2023-10-06T01:41:05.062553+0000","last_peered":"2023-10-06T01:41:05.062553+0000","last_clean":"2023-10-06T01:41:05.062553+0000","last_became_active":"2023-10-06T01:40:42.480165+0000","last_became_peered":"2023-10-06T01:40:42.480165+0000","last_unstale":"2023-10-06T01:41:05.062553+0000","last_undegraded":"2023-10-06T01:41:05.062553+0000","last_fullsized":"2023-10-06T01:41:05.062553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:58.858516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062556+0000","last_change":"2023-10-06T01:40:42.482708+0000","last_active":"2023-10-06T01:41:05.062556+0000","last_peered":"2023-10-06T01:41:05.062556+0000","last_clean":"2023-10-06T01:41:05.062556+0000","last_became_active":"2023-10-06T01:40:42.482408+0000","last_became_peered":"2023-10-06T01:40:42.482408+0000","last_unstale":"2023-10-06T01:41:05.062556+0000","last_undegraded":"2023-10-06T01:41:05.062556+0000","last_fullsized":"2023-10-06T01:41:05.062556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:04.066986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059961+0000","last_change":"2023-10-06T01:40:42.484414+0000","last_active":"2023-10-06T01:41:05.059961+0000","last_peered":"2023-10-06T01:41:05.059961+0000","last_clean":"2023-10-06T01:41:05.059961+0000","last_became_active":"2023-10-06T01:40:42.484163+0000","last_became_peered":"2023-10-06T01:40:42.484163+0000","last_unstale":"2023-10-06T01:41:05.059961+0000","last_undegraded":"2023-10-06T01:41:05.059961+0000","last_fullsized":"2023-10-06T01:41:05.059961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067584+0000","last_change":"2023-10-06T01:40:42.480695+0000","last_active":"2023-10-06T01:41:05.067584+0000","last_peered":"2023-10-06T01:41:05.067584+0000","last_clean":"2023-10-06T01:41:05.067584+0000","last_became_active":"2023-10-06T01:40:42.480454+0000","last_became_peered":"2023-10-06T01:40:42.480454+0000","last_unstale":"2023-10-06T01:41:05.067584+0000","last_undegraded":"2023-10-06T01:41:05.067584+0000","last_fullsized":"2023-10-06T01:41:05.067584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063813+0000","last_change":"2023-10-06T01:40:42.481147+0000","last_active":"2023-10-06T01:41:05.063813+0000","last_peered":"2023-10-06T01:41:05.063813+0000","last_clean":"2023-10-06T01:41:05.063813+0000","last_became_active":"2023-10-06T01:40:42.480844+0000","last_became_peered":"2023-10-06T01:40:42.480844+0000","last_unstale":"2023-10-06T01:41:05.063813+0000","last_undegraded":"2023-10-06T01:41:05.063813+0000","last_fullsized":"2023-10-06T01:41:05.063813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:40:42.472954+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:40:42.472431+0000","last_became_peered":"2023-10-06T01:40:42.472431+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.885339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.473497+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.473225+0000","last_became_peered":"2023-10-06T01:40:42.473225+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"36'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062072+0000","last_change":"2023-10-06T01:40:42.474680+0000","last_active":"2023-10-06T01:41:05.062072+0000","last_peered":"2023-10-06T01:41:05.062072+0000","last_clean":"2023-10-06T01:41:05.062072+0000","last_became_active":"2023-10-06T01:40:42.474460+0000","last_became_peered":"2023-10-06T01:40:42.474460+0000","last_unstale":"2023-10-06T01:41:05.062072+0000","last_undegraded":"2023-10-06T01:41:05.062072+0000","last_fullsized":"2023-10-06T01:41:05.062072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"36'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059914+0000","last_change":"2023-10-06T01:40:42.472843+0000","last_active":"2023-10-06T01:41:05.059914+0000","last_peered":"2023-10-06T01:41:05.059914+0000","last_clean":"2023-10-06T01:41:05.059914+0000","last_became_active":"2023-10-06T01:40:42.472588+0000","last_became_peered":"2023-10-06T01:40:42.472588+0000","last_unstale":"2023-10-06T01:41:05.059914+0000","last_undegraded":"2023-10-06T01:41:05.059914+0000","last_fullsized":"2023-10-06T01:41:05.059914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062784+0000","last_change":"2023-10-06T01:40:42.483391+0000","last_active":"2023-10-06T01:41:05.062784+0000","last_peered":"2023-10-06T01:41:05.062784+0000","last_clean":"2023-10-06T01:41:05.062784+0000","last_became_active":"2023-10-06T01:40:42.480152+0000","last_became_peered":"2023-10-06T01:40:42.480152+0000","last_unstale":"2023-10-06T01:41:05.062784+0000","last_undegraded":"2023-10-06T01:41:05.062784+0000","last_fullsized":"2023-10-06T01:41:05.062784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.896207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"36'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067606+0000","last_change":"2023-10-06T01:40:42.488522+0000","last_active":"2023-10-06T01:41:05.067606+0000","last_peered":"2023-10-06T01:41:05.067606+0000","last_clean":"2023-10-06T01:41:05.067606+0000","last_became_active":"2023-10-06T01:40:42.488255+0000","last_became_peered":"2023-10-06T01:40:42.488255+0000","last_unstale":"2023-10-06T01:41:05.067606+0000","last_undegraded":"2023-10-06T01:41:05.067606+0000","last_fullsized":"2023-10-06T01:41:05.067606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:53.772436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061786+0000","last_change":"2023-10-06T01:40:42.478517+0000","last_active":"2023-10-06T01:41:05.061786+0000","last_peered":"2023-10-06T01:41:05.061786+0000","last_clean":"2023-10-06T01:41:05.061786+0000","last_became_active":"2023-10-06T01:40:42.478229+0000","last_became_peered":"2023-10-06T01:40:42.478229+0000","last_unstale":"2023-10-06T01:41:05.061786+0000","last_undegraded":"2023-10-06T01:41:05.061786+0000","last_fullsized":"2023-10-06T01:41:05.061786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:46.493218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063049+0000","last_change":"2023-10-06T01:40:42.473146+0000","last_active":"2023-10-06T01:41:05.063049+0000","last_peered":"2023-10-06T01:41:05.063049+0000","last_clean":"2023-10-06T01:41:05.063049+0000","last_became_active":"2023-10-06T01:40:42.472562+0000","last_became_peered":"2023-10-06T01:40:42.472562+0000","last_unstale":"2023-10-06T01:41:05.063049+0000","last_undegraded":"2023-10-06T01:41:05.063049+0000","last_fullsized":"2023-10-06T01:41:05.063049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"36'12","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066771+0000","last_change":"2023-10-06T01:40:42.480272+0000","last_active":"2023-10-06T01:41:05.066771+0000","last_peered":"2023-10-06T01:41:05.066771+0000","last_clean":"2023-10-06T01:41:05.066771+0000","last_became_active":"2023-10-06T01:40:42.480015+0000","last_became_peered":"2023-10-06T01:40:42.480015+0000","last_unstale":"2023-10-06T01:41:05.066771+0000","last_undegraded":"2023-10-06T01:41:05.066771+0000","last_fullsized":"2023-10-06T01:41:05.066771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059829+0000","last_change":"2023-10-06T01:40:42.479462+0000","last_active":"2023-10-06T01:41:05.059829+0000","last_peered":"2023-10-06T01:41:05.059829+0000","last_clean":"2023-10-06T01:41:05.059829+0000","last_became_active":"2023-10-06T01:40:42.479195+0000","last_became_peered":"2023-10-06T01:40:42.479195+0000","last_unstale":"2023-10-06T01:41:05.059829+0000","last_undegraded":"2023-10-06T01:41:05.059829+0000","last_fullsized":"2023-10-06T01:41:05.059829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:36.952749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074919+0000","last_change":"2023-10-06T01:41:05.072432+0000","last_active":"2023-10-06T01:41:05.074919+0000","last_peered":"2023-10-06T01:41:05.074919+0000","last_clean":"2023-10-06T01:41:05.074919+0000","last_became_active":"2023-10-06T01:41:05.072154+0000","last_became_peered":"2023-10-06T01:41:05.072154+0000","last_unstale":"2023-10-06T01:41:05.074919+0000","last_undegraded":"2023-10-06T01:41:05.074919+0000","last_fullsized":"2023-10-06T01:41:05.074919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:55.047887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063306+0000","last_change":"2023-10-06T01:40:42.471146+0000","last_active":"2023-10-06T01:41:05.063306+0000","last_peered":"2023-10-06T01:41:05.063306+0000","last_clean":"2023-10-06T01:41:05.063306+0000","last_became_active":"2023-10-06T01:40:42.470886+0000","last_became_peered":"2023-10-06T01:40:42.470886+0000","last_unstale":"2023-10-06T01:41:05.063306+0000","last_undegraded":"2023-10-06T01:41:05.063306+0000","last_fullsized":"2023-10-06T01:41:05.063306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068615+0000","last_change":"2023-10-06T01:40:41.467749+0000","last_active":"2023-10-06T01:41:05.068615+0000","last_peered":"2023-10-06T01:41:05.068615+0000","last_clean":"2023-10-06T01:41:05.068615+0000","last_became_active":"2023-10-06T01:40:41.467482+0000","last_became_peered":"2023-10-06T01:40:41.467482+0000","last_unstale":"2023-10-06T01:41:05.068615+0000","last_undegraded":"2023-10-06T01:41:05.068615+0000","last_fullsized":"2023-10-06T01:41:05.068615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077358+0000","last_change":"2023-10-06T01:41:05.075392+0000","last_active":"2023-10-06T01:41:05.077358+0000","last_peered":"2023-10-06T01:41:05.077358+0000","last_clean":"2023-10-06T01:41:05.077358+0000","last_became_active":"2023-10-06T01:41:05.075108+0000","last_became_peered":"2023-10-06T01:41:05.075108+0000","last_unstale":"2023-10-06T01:41:05.077358+0000","last_undegraded":"2023-10-06T01:41:05.077358+0000","last_fullsized":"2023-10-06T01:41:05.077358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:43.179045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061850+0000","last_change":"2023-10-06T01:40:42.465919+0000","last_active":"2023-10-06T01:41:05.061850+0000","last_peered":"2023-10-06T01:41:05.061850+0000","last_clean":"2023-10-06T01:41:05.061850+0000","last_became_active":"2023-10-06T01:40:42.465701+0000","last_became_peered":"2023-10-06T01:40:42.465701+0000","last_unstale":"2023-10-06T01:41:05.061850+0000","last_undegraded":"2023-10-06T01:41:05.061850+0000","last_fullsized":"2023-10-06T01:41:05.061850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:07.779153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063096+0000","last_change":"2023-10-06T01:40:41.472783+0000","last_active":"2023-10-06T01:41:05.063096+0000","last_peered":"2023-10-06T01:41:05.063096+0000","last_clean":"2023-10-06T01:41:05.063096+0000","last_became_active":"2023-10-06T01:40:41.472534+0000","last_became_peered":"2023-10-06T01:40:41.472534+0000","last_unstale":"2023-10-06T01:41:05.063096+0000","last_undegraded":"2023-10-06T01:41:05.063096+0000","last_fullsized":"2023-10-06T01:41:05.063096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:10.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075586+0000","last_change":"2023-10-06T01:41:05.072321+0000","last_active":"2023-10-06T01:41:05.075586+0000","last_peered":"2023-10-06T01:41:05.075586+0000","last_clean":"2023-10-06T01:41:05.075586+0000","last_became_active":"2023-10-06T01:41:05.072037+0000","last_became_peered":"2023-10-06T01:41:05.072037+0000","last_unstale":"2023-10-06T01:41:05.075586+0000","last_undegraded":"2023-10-06T01:41:05.075586+0000","last_fullsized":"2023-10-06T01:41:05.075586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067494+0000","last_change":"2023-10-06T01:40:42.485536+0000","last_active":"2023-10-06T01:41:05.067494+0000","last_peered":"2023-10-06T01:41:05.067494+0000","last_clean":"2023-10-06T01:41:05.067494+0000","last_became_active":"2023-10-06T01:40:42.485269+0000","last_became_peered":"2023-10-06T01:40:42.485269+0000","last_unstale":"2023-10-06T01:41:05.067494+0000","last_undegraded":"2023-10-06T01:41:05.067494+0000","last_fullsized":"2023-10-06T01:41:05.067494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:24.452066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062861+0000","last_change":"2023-10-06T01:40:41.470447+0000","last_active":"2023-10-06T01:41:05.062861+0000","last_peered":"2023-10-06T01:41:05.062861+0000","last_clean":"2023-10-06T01:41:05.062861+0000","last_became_active":"2023-10-06T01:40:41.470193+0000","last_became_peered":"2023-10-06T01:40:41.470193+0000","last_unstale":"2023-10-06T01:41:05.062861+0000","last_undegraded":"2023-10-06T01:41:05.062861+0000","last_fullsized":"2023-10-06T01:41:05.062861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:01:45.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071654+0000","last_change":"2023-10-06T01:41:05.070017+0000","last_active":"2023-10-06T01:41:05.071654+0000","last_peered":"2023-10-06T01:41:05.071654+0000","last_clean":"2023-10-06T01:41:05.071654+0000","last_became_active":"2023-10-06T01:41:05.069741+0000","last_became_peered":"2023-10-06T01:41:05.069741+0000","last_unstale":"2023-10-06T01:41:05.071654+0000","last_undegraded":"2023-10-06T01:41:05.071654+0000","last_fullsized":"2023-10-06T01:41:05.071654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:36.570717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062619+0000","last_change":"2023-10-06T01:40:42.471612+0000","last_active":"2023-10-06T01:41:05.062619+0000","last_peered":"2023-10-06T01:41:05.062619+0000","last_clean":"2023-10-06T01:41:05.062619+0000","last_became_active":"2023-10-06T01:40:42.471356+0000","last_became_peered":"2023-10-06T01:40:42.471356+0000","last_unstale":"2023-10-06T01:41:05.062619+0000","last_undegraded":"2023-10-06T01:41:05.062619+0000","last_fullsized":"2023-10-06T01:41:05.062619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067659+0000","last_change":"2023-10-06T01:40:41.468343+0000","last_active":"2023-10-06T01:41:05.067659+0000","last_peered":"2023-10-06T01:41:05.067659+0000","last_clean":"2023-10-06T01:41:05.067659+0000","last_became_active":"2023-10-06T01:40:41.468065+0000","last_became_peered":"2023-10-06T01:40:41.468065+0000","last_unstale":"2023-10-06T01:41:05.067659+0000","last_undegraded":"2023-10-06T01:41:05.067659+0000","last_fullsized":"2023-10-06T01:41:05.067659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:28.228515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077336+0000","last_change":"2023-10-06T01:41:05.074701+0000","last_active":"2023-10-06T01:41:05.077336+0000","last_peered":"2023-10-06T01:41:05.077336+0000","last_clean":"2023-10-06T01:41:05.077336+0000","last_became_active":"2023-10-06T01:41:05.074479+0000","last_became_peered":"2023-10-06T01:41:05.074479+0000","last_unstale":"2023-10-06T01:41:05.077336+0000","last_undegraded":"2023-10-06T01:41:05.077336+0000","last_fullsized":"2023-10-06T01:41:05.077336+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:42.689091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063152+0000","last_change":"2023-10-06T01:40:42.473318+0000","last_active":"2023-10-06T01:41:05.063152+0000","last_peered":"2023-10-06T01:41:05.063152+0000","last_clean":"2023-10-06T01:41:05.063152+0000","last_became_active":"2023-10-06T01:40:42.472899+0000","last_became_peered":"2023-10-06T01:40:42.472899+0000","last_unstale":"2023-10-06T01:41:05.063152+0000","last_undegraded":"2023-10-06T01:41:05.063152+0000","last_fullsized":"2023-10-06T01:41:05.063152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:10.027552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067927+0000","last_change":"2023-10-06T01:40:41.464311+0000","last_active":"2023-10-06T01:41:05.067927+0000","last_peered":"2023-10-06T01:41:05.067927+0000","last_clean":"2023-10-06T01:41:05.067927+0000","last_became_active":"2023-10-06T01:40:41.464070+0000","last_became_peered":"2023-10-06T01:40:41.464070+0000","last_unstale":"2023-10-06T01:41:05.067927+0000","last_undegraded":"2023-10-06T01:41:05.067927+0000","last_fullsized":"2023-10-06T01:41:05.067927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:05.458001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071094+0000","last_change":"2023-10-06T01:41:05.068576+0000","last_active":"2023-10-06T01:41:05.071094+0000","last_peered":"2023-10-06T01:41:05.071094+0000","last_clean":"2023-10-06T01:41:05.071094+0000","last_became_active":"2023-10-06T01:41:05.068329+0000","last_became_peered":"2023-10-06T01:41:05.068329+0000","last_unstale":"2023-10-06T01:41:05.071094+0000","last_undegraded":"2023-10-06T01:41:05.071094+0000","last_fullsized":"2023-10-06T01:41:05.071094+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063824+0000","last_change":"2023-10-06T01:40:42.475400+0000","last_active":"2023-10-06T01:41:05.063824+0000","last_peered":"2023-10-06T01:41:05.063824+0000","last_clean":"2023-10-06T01:41:05.063824+0000","last_became_active":"2023-10-06T01:40:42.475120+0000","last_became_peered":"2023-10-06T01:40:42.475120+0000","last_unstale":"2023-10-06T01:41:05.063824+0000","last_undegraded":"2023-10-06T01:41:05.063824+0000","last_fullsized":"2023-10-06T01:41:05.063824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063903+0000","last_change":"2023-10-06T01:40:41.468752+0000","last_active":"2023-10-06T01:41:05.063903+0000","last_peered":"2023-10-06T01:41:05.063903+0000","last_clean":"2023-10-06T01:41:05.063903+0000","last_became_active":"2023-10-06T01:40:41.468317+0000","last_became_peered":"2023-10-06T01:40:41.468317+0000","last_unstale":"2023-10-06T01:41:05.063903+0000","last_undegraded":"2023-10-06T01:41:05.063903+0000","last_fullsized":"2023-10-06T01:41:05.063903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:33.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.076999+0000","last_change":"2023-10-06T01:41:05.072828+0000","last_active":"2023-10-06T01:41:05.076999+0000","last_peered":"2023-10-06T01:41:05.076999+0000","last_clean":"2023-10-06T01:41:05.076999+0000","last_became_active":"2023-10-06T01:41:05.072570+0000","last_became_peered":"2023-10-06T01:41:05.072570+0000","last_unstale":"2023-10-06T01:41:05.076999+0000","last_undegraded":"2023-10-06T01:41:05.076999+0000","last_fullsized":"2023-10-06T01:41:05.076999+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:05.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064241+0000","last_change":"2023-10-06T01:40:42.478912+0000","last_active":"2023-10-06T01:41:05.064241+0000","last_peered":"2023-10-06T01:41:05.064241+0000","last_clean":"2023-10-06T01:41:05.064241+0000","last_became_active":"2023-10-06T01:40:42.478673+0000","last_became_peered":"2023-10-06T01:40:42.478673+0000","last_unstale":"2023-10-06T01:41:05.064241+0000","last_undegraded":"2023-10-06T01:41:05.064241+0000","last_fullsized":"2023-10-06T01:41:05.064241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:30.845017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066774+0000","last_change":"2023-10-06T01:40:41.467878+0000","last_active":"2023-10-06T01:41:05.066774+0000","last_peered":"2023-10-06T01:41:05.066774+0000","last_clean":"2023-10-06T01:41:05.066774+0000","last_became_active":"2023-10-06T01:40:41.467618+0000","last_became_peered":"2023-10-06T01:40:41.467618+0000","last_unstale":"2023-10-06T01:41:05.066774+0000","last_undegraded":"2023-10-06T01:41:05.066774+0000","last_fullsized":"2023-10-06T01:41:05.066774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077743+0000","last_change":"2023-10-06T01:41:05.075317+0000","last_active":"2023-10-06T01:41:05.077743+0000","last_peered":"2023-10-06T01:41:05.077743+0000","last_clean":"2023-10-06T01:41:05.077743+0000","last_became_active":"2023-10-06T01:41:05.075139+0000","last_became_peered":"2023-10-06T01:41:05.075139+0000","last_unstale":"2023-10-06T01:41:05.077743+0000","last_undegraded":"2023-10-06T01:41:05.077743+0000","last_fullsized":"2023-10-06T01:41:05.077743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062926+0000","last_change":"2023-10-06T01:40:42.473067+0000","last_active":"2023-10-06T01:41:05.062926+0000","last_peered":"2023-10-06T01:41:05.062926+0000","last_clean":"2023-10-06T01:41:05.062926+0000","last_became_active":"2023-10-06T01:40:42.472808+0000","last_became_peered":"2023-10-06T01:40:42.472808+0000","last_unstale":"2023-10-06T01:41:05.062926+0000","last_undegraded":"2023-10-06T01:41:05.062926+0000","last_fullsized":"2023-10-06T01:41:05.062926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:20.630531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064314+0000","last_change":"2023-10-06T01:40:41.466031+0000","last_active":"2023-10-06T01:41:05.064314+0000","last_peered":"2023-10-06T01:41:05.064314+0000","last_clean":"2023-10-06T01:41:05.064314+0000","last_became_active":"2023-10-06T01:40:41.465767+0000","last_became_peered":"2023-10-06T01:40:41.465767+0000","last_unstale":"2023-10-06T01:41:05.064314+0000","last_undegraded":"2023-10-06T01:41:05.064314+0000","last_fullsized":"2023-10-06T01:41:05.064314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074604+0000","last_change":"2023-10-06T01:41:05.069137+0000","last_active":"2023-10-06T01:41:05.074604+0000","last_peered":"2023-10-06T01:41:05.074604+0000","last_clean":"2023-10-06T01:41:05.074604+0000","last_became_active":"2023-10-06T01:41:05.068861+0000","last_became_peered":"2023-10-06T01:41:05.068861+0000","last_unstale":"2023-10-06T01:41:05.074604+0000","last_undegraded":"2023-10-06T01:41:05.074604+0000","last_fullsized":"2023-10-06T01:41:05.074604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:54.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063296+0000","last_change":"2023-10-06T01:40:42.487339+0000","last_active":"2023-10-06T01:41:05.063296+0000","last_peered":"2023-10-06T01:41:05.063296+0000","last_clean":"2023-10-06T01:41:05.063296+0000","last_became_active":"2023-10-06T01:40:42.487044+0000","last_became_peered":"2023-10-06T01:40:42.487044+0000","last_unstale":"2023-10-06T01:41:05.063296+0000","last_undegraded":"2023-10-06T01:41:05.063296+0000","last_fullsized":"2023-10-06T01:41:05.063296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:32.646521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063779+0000","last_change":"2023-10-06T01:40:41.463319+0000","last_active":"2023-10-06T01:41:05.063779+0000","last_peered":"2023-10-06T01:41:05.063779+0000","last_clean":"2023-10-06T01:41:05.063779+0000","last_became_active":"2023-10-06T01:40:41.463096+0000","last_became_peered":"2023-10-06T01:40:41.463096+0000","last_unstale":"2023-10-06T01:41:05.063779+0000","last_undegraded":"2023-10-06T01:41:05.063779+0000","last_fullsized":"2023-10-06T01:41:05.063779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:55:52.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073651+0000","last_change":"2023-10-06T01:41:05.071706+0000","last_active":"2023-10-06T01:41:05.073651+0000","last_peered":"2023-10-06T01:41:05.073651+0000","last_clean":"2023-10-06T01:41:05.073651+0000","last_became_active":"2023-10-06T01:41:05.071432+0000","last_became_peered":"2023-10-06T01:41:05.071432+0000","last_unstale":"2023-10-06T01:41:05.073651+0000","last_undegraded":"2023-10-06T01:41:05.073651+0000","last_fullsized":"2023-10-06T01:41:05.073651+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070266+0000","last_change":"2023-10-06T01:40:42.483351+0000","last_active":"2023-10-06T01:41:05.070266+0000","last_peered":"2023-10-06T01:41:05.070266+0000","last_clean":"2023-10-06T01:41:05.070266+0000","last_became_active":"2023-10-06T01:40:42.483078+0000","last_became_peered":"2023-10-06T01:40:42.483078+0000","last_unstale":"2023-10-06T01:41:05.070266+0000","last_undegraded":"2023-10-06T01:41:05.070266+0000","last_fullsized":"2023-10-06T01:41:05.070266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:07.421010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067322+0000","last_change":"2023-10-06T01:40:41.469093+0000","last_active":"2023-10-06T01:41:05.067322+0000","last_peered":"2023-10-06T01:41:05.067322+0000","last_clean":"2023-10-06T01:41:05.067322+0000","last_became_active":"2023-10-06T01:40:41.468790+0000","last_became_peered":"2023-10-06T01:40:41.468790+0000","last_unstale":"2023-10-06T01:41:05.067322+0000","last_undegraded":"2023-10-06T01:41:05.067322+0000","last_fullsized":"2023-10-06T01:41:05.067322+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.880644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070176+0000","last_change":"2023-10-06T01:41:05.066490+0000","last_active":"2023-10-06T01:41:05.070176+0000","last_peered":"2023-10-06T01:41:05.070176+0000","last_clean":"2023-10-06T01:41:05.070176+0000","last_became_active":"2023-10-06T01:41:05.065985+0000","last_became_peered":"2023-10-06T01:41:05.065985+0000","last_unstale":"2023-10-06T01:41:05.070176+0000","last_undegraded":"2023-10-06T01:41:05.070176+0000","last_fullsized":"2023-10-06T01:41:05.070176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"36'11","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067405+0000","last_change":"2023-10-06T01:40:42.476048+0000","last_active":"2023-10-06T01:41:05.067405+0000","last_peered":"2023-10-06T01:41:05.067405+0000","last_clean":"2023-10-06T01:41:05.067405+0000","last_became_active":"2023-10-06T01:40:42.475616+0000","last_became_peered":"2023-10-06T01:40:42.475616+0000","last_unstale":"2023-10-06T01:41:05.067405+0000","last_undegraded":"2023-10-06T01:41:05.067405+0000","last_fullsized":"2023-10-06T01:41:05.067405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:14.048255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069275+0000","last_change":"2023-10-06T01:40:41.472661+0000","last_active":"2023-10-06T01:41:05.069275+0000","last_peered":"2023-10-06T01:41:05.069275+0000","last_clean":"2023-10-06T01:41:05.069275+0000","last_became_active":"2023-10-06T01:40:41.472389+0000","last_became_peered":"2023-10-06T01:40:41.472389+0000","last_unstale":"2023-10-06T01:41:05.069275+0000","last_undegraded":"2023-10-06T01:41:05.069275+0000","last_fullsized":"2023-10-06T01:41:05.069275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:35.948054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074516+0000","last_change":"2023-10-06T01:41:05.071088+0000","last_active":"2023-10-06T01:41:05.074516+0000","last_peered":"2023-10-06T01:41:05.074516+0000","last_clean":"2023-10-06T01:41:05.074516+0000","last_became_active":"2023-10-06T01:41:05.070727+0000","last_became_peered":"2023-10-06T01:41:05.070727+0000","last_unstale":"2023-10-06T01:41:05.074516+0000","last_undegraded":"2023-10-06T01:41:05.074516+0000","last_fullsized":"2023-10-06T01:41:05.074516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.767671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068853+0000","last_change":"2023-10-06T01:40:42.478379+0000","last_active":"2023-10-06T01:41:05.068853+0000","last_peered":"2023-10-06T01:41:05.068853+0000","last_clean":"2023-10-06T01:41:05.068853+0000","last_became_active":"2023-10-06T01:40:42.476879+0000","last_became_peered":"2023-10-06T01:40:42.476879+0000","last_unstale":"2023-10-06T01:41:05.068853+0000","last_undegraded":"2023-10-06T01:41:05.068853+0000","last_fullsized":"2023-10-06T01:41:05.068853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:26.687647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'22","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060056+0000","last_change":"2023-10-06T01:40:41.465691+0000","last_active":"2023-10-06T01:41:05.060056+0000","last_peered":"2023-10-06T01:41:05.060056+0000","last_clean":"2023-10-06T01:41:05.060056+0000","last_became_active":"2023-10-06T01:40:41.465429+0000","last_became_peered":"2023-10-06T01:40:41.465429+0000","last_unstale":"2023-10-06T01:41:05.060056+0000","last_undegraded":"2023-10-06T01:41:05.060056+0000","last_fullsized":"2023-10-06T01:41:05.060056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:22.750907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077429+0000","last_change":"2023-10-06T01:41:05.072554+0000","last_active":"2023-10-06T01:41:05.077429+0000","last_peered":"2023-10-06T01:41:05.077429+0000","last_clean":"2023-10-06T01:41:05.077429+0000","last_became_active":"2023-10-06T01:41:05.072311+0000","last_became_peered":"2023-10-06T01:41:05.072311+0000","last_unstale":"2023-10-06T01:41:05.077429+0000","last_undegraded":"2023-10-06T01:41:05.077429+0000","last_fullsized":"2023-10-06T01:41:05.077429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"36'10","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062171+0000","last_change":"2023-10-06T01:40:42.484065+0000","last_active":"2023-10-06T01:41:05.062171+0000","last_peered":"2023-10-06T01:41:05.062171+0000","last_clean":"2023-10-06T01:41:05.062171+0000","last_became_active":"2023-10-06T01:40:42.483767+0000","last_became_peered":"2023-10-06T01:40:42.483767+0000","last_unstale":"2023-10-06T01:41:05.062171+0000","last_undegraded":"2023-10-06T01:41:05.062171+0000","last_fullsized":"2023-10-06T01:41:05.062171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:22.153394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067767+0000","last_change":"2023-10-06T01:40:41.468974+0000","last_active":"2023-10-06T01:41:05.067767+0000","last_peered":"2023-10-06T01:41:05.067767+0000","last_clean":"2023-10-06T01:41:05.067767+0000","last_became_active":"2023-10-06T01:40:41.468474+0000","last_became_peered":"2023-10-06T01:40:41.468474+0000","last_unstale":"2023-10-06T01:41:05.067767+0000","last_undegraded":"2023-10-06T01:41:05.067767+0000","last_fullsized":"2023-10-06T01:41:05.067767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.514210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074931+0000","last_change":"2023-10-06T01:41:05.072492+0000","last_active":"2023-10-06T01:41:05.074931+0000","last_peered":"2023-10-06T01:41:05.074931+0000","last_clean":"2023-10-06T01:41:05.074931+0000","last_became_active":"2023-10-06T01:41:05.072225+0000","last_became_peered":"2023-10-06T01:41:05.072225+0000","last_unstale":"2023-10-06T01:41:05.074931+0000","last_undegraded":"2023-10-06T01:41:05.074931+0000","last_fullsized":"2023-10-06T01:41:05.074931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.539127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"36'15","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063929+0000","last_change":"2023-10-06T01:40:42.478175+0000","last_active":"2023-10-06T01:41:05.063929+0000","last_peered":"2023-10-06T01:41:05.063929+0000","last_clean":"2023-10-06T01:41:05.063929+0000","last_became_active":"2023-10-06T01:40:42.477913+0000","last_became_peered":"2023-10-06T01:40:42.477913+0000","last_unstale":"2023-10-06T01:41:05.063929+0000","last_undegraded":"2023-10-06T01:41:05.063929+0000","last_fullsized":"2023-10-06T01:41:05.063929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067895+0000","last_change":"2023-10-06T01:40:41.466657+0000","last_active":"2023-10-06T01:41:05.067895+0000","last_peered":"2023-10-06T01:41:05.067895+0000","last_clean":"2023-10-06T01:41:05.067895+0000","last_became_active":"2023-10-06T01:40:41.466394+0000","last_became_peered":"2023-10-06T01:40:41.466394+0000","last_unstale":"2023-10-06T01:41:05.067895+0000","last_undegraded":"2023-10-06T01:41:05.067895+0000","last_fullsized":"2023-10-06T01:41:05.067895+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:04.847000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077494+0000","last_change":"2023-10-06T01:41:05.072597+0000","last_active":"2023-10-06T01:41:05.077494+0000","last_peered":"2023-10-06T01:41:05.077494+0000","last_clean":"2023-10-06T01:41:05.077494+0000","last_became_active":"2023-10-06T01:41:05.072379+0000","last_became_peered":"2023-10-06T01:41:05.072379+0000","last_unstale":"2023-10-06T01:41:05.077494+0000","last_undegraded":"2023-10-06T01:41:05.077494+0000","last_fullsized":"2023-10-06T01:41:05.077494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"36'6","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067386+0000","last_change":"2023-10-06T01:40:42.473815+0000","last_active":"2023-10-06T01:41:05.067386+0000","last_peered":"2023-10-06T01:41:05.067386+0000","last_clean":"2023-10-06T01:41:05.067386+0000","last_became_active":"2023-10-06T01:40:42.473480+0000","last_became_peered":"2023-10-06T01:40:42.473480+0000","last_unstale":"2023-10-06T01:41:05.067386+0000","last_undegraded":"2023-10-06T01:41:05.067386+0000","last_fullsized":"2023-10-06T01:41:05.067386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:35.340264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066926+0000","last_change":"2023-10-06T01:40:41.467708+0000","last_active":"2023-10-06T01:41:05.066926+0000","last_peered":"2023-10-06T01:41:05.066926+0000","last_clean":"2023-10-06T01:41:05.066926+0000","last_became_active":"2023-10-06T01:40:41.467442+0000","last_became_peered":"2023-10-06T01:40:41.467442+0000","last_unstale":"2023-10-06T01:41:05.066926+0000","last_undegraded":"2023-10-06T01:41:05.066926+0000","last_fullsized":"2023-10-06T01:41:05.066926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:00.308681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.073491+0000","last_change":"2023-10-06T01:41:05.070833+0000","last_active":"2023-10-06T01:41:05.073491+0000","last_peered":"2023-10-06T01:41:05.073491+0000","last_clean":"2023-10-06T01:41:05.073491+0000","last_became_active":"2023-10-06T01:41:05.070486+0000","last_became_peered":"2023-10-06T01:41:05.070486+0000","last_unstale":"2023-10-06T01:41:05.073491+0000","last_undegraded":"2023-10-06T01:41:05.073491+0000","last_fullsized":"2023-10-06T01:41:05.073491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:26:41.956291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"36'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068137+0000","last_change":"2023-10-06T01:40:42.481908+0000","last_active":"2023-10-06T01:41:05.068137+0000","last_peered":"2023-10-06T01:41:05.068137+0000","last_clean":"2023-10-06T01:41:05.068137+0000","last_became_active":"2023-10-06T01:40:42.481606+0000","last_became_peered":"2023-10-06T01:40:42.481606+0000","last_unstale":"2023-10-06T01:41:05.068137+0000","last_undegraded":"2023-10-06T01:41:05.068137+0000","last_fullsized":"2023-10-06T01:41:05.068137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:36.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'15","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064395+0000","last_change":"2023-10-06T01:40:41.469840+0000","last_active":"2023-10-06T01:41:05.064395+0000","last_peered":"2023-10-06T01:41:05.064395+0000","last_clean":"2023-10-06T01:41:05.064395+0000","last_became_active":"2023-10-06T01:40:41.469129+0000","last_became_peered":"2023-10-06T01:40:41.469129+0000","last_unstale":"2023-10-06T01:41:05.064395+0000","last_undegraded":"2023-10-06T01:41:05.064395+0000","last_fullsized":"2023-10-06T01:41:05.064395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:06.774457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071482+0000","last_change":"2023-10-06T01:41:05.069668+0000","last_active":"2023-10-06T01:41:05.071482+0000","last_peered":"2023-10-06T01:41:05.071482+0000","last_clean":"2023-10-06T01:41:05.071482+0000","last_became_active":"2023-10-06T01:41:05.069033+0000","last_became_peered":"2023-10-06T01:41:05.069033+0000","last_unstale":"2023-10-06T01:41:05.071482+0000","last_undegraded":"2023-10-06T01:41:05.071482+0000","last_fullsized":"2023-10-06T01:41:05.071482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:14:32.941476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059936+0000","last_change":"2023-10-06T01:40:42.484218+0000","last_active":"2023-10-06T01:41:05.059936+0000","last_peered":"2023-10-06T01:41:05.059936+0000","last_clean":"2023-10-06T01:41:05.059936+0000","last_became_active":"2023-10-06T01:40:42.483982+0000","last_became_peered":"2023-10-06T01:40:42.483982+0000","last_unstale":"2023-10-06T01:41:05.059936+0000","last_undegraded":"2023-10-06T01:41:05.059936+0000","last_fullsized":"2023-10-06T01:41:05.059936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:29.233211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'2","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.060008+0000","last_change":"2023-10-06T01:40:41.466682+0000","last_active":"2023-10-06T01:41:05.060008+0000","last_peered":"2023-10-06T01:41:05.060008+0000","last_clean":"2023-10-06T01:41:05.060008+0000","last_became_active":"2023-10-06T01:40:41.466451+0000","last_became_peered":"2023-10-06T01:40:41.466451+0000","last_unstale":"2023-10-06T01:41:05.060008+0000","last_undegraded":"2023-10-06T01:41:05.060008+0000","last_fullsized":"2023-10-06T01:41:05.060008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:07.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075650+0000","last_change":"2023-10-06T01:41:05.072868+0000","last_active":"2023-10-06T01:41:05.075650+0000","last_peered":"2023-10-06T01:41:05.075650+0000","last_clean":"2023-10-06T01:41:05.075650+0000","last_became_active":"2023-10-06T01:41:05.072621+0000","last_became_peered":"2023-10-06T01:41:05.072621+0000","last_unstale":"2023-10-06T01:41:05.075650+0000","last_undegraded":"2023-10-06T01:41:05.075650+0000","last_fullsized":"2023-10-06T01:41:05.075650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067784+0000","last_change":"2023-10-06T01:40:42.483508+0000","last_active":"2023-10-06T01:41:05.067784+0000","last_peered":"2023-10-06T01:41:05.067784+0000","last_clean":"2023-10-06T01:41:05.067784+0000","last_became_active":"2023-10-06T01:40:42.483227+0000","last_became_peered":"2023-10-06T01:40:42.483227+0000","last_unstale":"2023-10-06T01:41:05.067784+0000","last_undegraded":"2023-10-06T01:41:05.067784+0000","last_fullsized":"2023-10-06T01:41:05.067784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.277321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'3","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063389+0000","last_change":"2023-10-06T01:40:41.465545+0000","last_active":"2023-10-06T01:41:05.063389+0000","last_peered":"2023-10-06T01:41:05.063389+0000","last_clean":"2023-10-06T01:41:05.063389+0000","last_became_active":"2023-10-06T01:40:41.465338+0000","last_became_peered":"2023-10-06T01:40:41.465338+0000","last_unstale":"2023-10-06T01:41:05.063389+0000","last_undegraded":"2023-10-06T01:41:05.063389+0000","last_fullsized":"2023-10-06T01:41:05.063389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:06.416315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.080676+0000","last_change":"2023-10-06T01:41:05.073304+0000","last_active":"2023-10-06T01:41:05.080676+0000","last_peered":"2023-10-06T01:41:05.080676+0000","last_clean":"2023-10-06T01:41:05.080676+0000","last_became_active":"2023-10-06T01:41:05.073056+0000","last_became_peered":"2023-10-06T01:41:05.073056+0000","last_unstale":"2023-10-06T01:41:05.080676+0000","last_undegraded":"2023-10-06T01:41:05.080676+0000","last_fullsized":"2023-10-06T01:41:05.080676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:14.536410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"36'6","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.061740+0000","last_change":"2023-10-06T01:40:42.474713+0000","last_active":"2023-10-06T01:41:05.061740+0000","last_peered":"2023-10-06T01:41:05.061740+0000","last_clean":"2023-10-06T01:41:05.061740+0000","last_became_active":"2023-10-06T01:40:42.474365+0000","last_became_peered":"2023-10-06T01:40:42.474365+0000","last_unstale":"2023-10-06T01:41:05.061740+0000","last_undegraded":"2023-10-06T01:41:05.061740+0000","last_fullsized":"2023-10-06T01:41:05.061740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:05.851696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'2","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067860+0000","last_change":"2023-10-06T01:40:41.468411+0000","last_active":"2023-10-06T01:41:05.067860+0000","last_peered":"2023-10-06T01:41:05.067860+0000","last_clean":"2023-10-06T01:41:05.067860+0000","last_became_active":"2023-10-06T01:40:41.468113+0000","last_became_peered":"2023-10-06T01:40:41.468113+0000","last_unstale":"2023-10-06T01:41:05.067860+0000","last_undegraded":"2023-10-06T01:41:05.067860+0000","last_fullsized":"2023-10-06T01:41:05.067860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:03.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068813+0000","last_change":"2023-10-06T01:41:05.066269+0000","last_active":"2023-10-06T01:41:05.068813+0000","last_peered":"2023-10-06T01:41:05.068813+0000","last_clean":"2023-10-06T01:41:05.068813+0000","last_became_active":"2023-10-06T01:41:05.065979+0000","last_became_peered":"2023-10-06T01:41:05.065979+0000","last_unstale":"2023-10-06T01:41:05.068813+0000","last_undegraded":"2023-10-06T01:41:05.068813+0000","last_fullsized":"2023-10-06T01:41:05.068813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:09.437261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"36'7","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062811+0000","last_change":"2023-10-06T01:40:42.471507+0000","last_active":"2023-10-06T01:41:05.062811+0000","last_peered":"2023-10-06T01:41:05.062811+0000","last_clean":"2023-10-06T01:41:05.062811+0000","last_became_active":"2023-10-06T01:40:42.471246+0000","last_became_peered":"2023-10-06T01:40:42.471246+0000","last_unstale":"2023-10-06T01:41:05.062811+0000","last_undegraded":"2023-10-06T01:41:05.062811+0000","last_fullsized":"2023-10-06T01:41:05.062811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:30.325663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067544+0000","last_change":"2023-10-06T01:40:41.468147+0000","last_active":"2023-10-06T01:41:05.067544+0000","last_peered":"2023-10-06T01:41:05.067544+0000","last_clean":"2023-10-06T01:41:05.067544+0000","last_became_active":"2023-10-06T01:40:41.467953+0000","last_became_peered":"2023-10-06T01:40:41.467953+0000","last_unstale":"2023-10-06T01:41:05.067544+0000","last_undegraded":"2023-10-06T01:41:05.067544+0000","last_fullsized":"2023-10-06T01:41:05.067544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:51.922611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.070058+0000","last_change":"2023-10-06T01:41:05.064865+0000","last_active":"2023-10-06T01:41:05.070058+0000","last_peered":"2023-10-06T01:41:05.070058+0000","last_clean":"2023-10-06T01:41:05.070058+0000","last_became_active":"2023-10-06T01:41:05.064644+0000","last_became_peered":"2023-10-06T01:41:05.064644+0000","last_unstale":"2023-10-06T01:41:05.070058+0000","last_undegraded":"2023-10-06T01:41:05.070058+0000","last_fullsized":"2023-10-06T01:41:05.070058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:27.127112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"36'5","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067040+0000","last_change":"2023-10-06T01:40:42.473907+0000","last_active":"2023-10-06T01:41:05.067040+0000","last_peered":"2023-10-06T01:41:05.067040+0000","last_clean":"2023-10-06T01:41:05.067040+0000","last_became_active":"2023-10-06T01:40:42.473587+0000","last_became_peered":"2023-10-06T01:40:42.473587+0000","last_unstale":"2023-10-06T01:41:05.067040+0000","last_undegraded":"2023-10-06T01:41:05.067040+0000","last_fullsized":"2023-10-06T01:41:05.067040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'4","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067848+0000","last_change":"2023-10-06T01:40:41.464033+0000","last_active":"2023-10-06T01:41:05.067848+0000","last_peered":"2023-10-06T01:41:05.067848+0000","last_clean":"2023-10-06T01:41:05.067848+0000","last_became_active":"2023-10-06T01:40:41.463751+0000","last_became_peered":"2023-10-06T01:40:41.463751+0000","last_unstale":"2023-10-06T01:41:05.067848+0000","last_undegraded":"2023-10-06T01:41:05.067848+0000","last_fullsized":"2023-10-06T01:41:05.067848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:25.682952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074693+0000","last_change":"2023-10-06T01:41:05.069995+0000","last_active":"2023-10-06T01:41:05.074693+0000","last_peered":"2023-10-06T01:41:05.074693+0000","last_clean":"2023-10-06T01:41:05.074693+0000","last_became_active":"2023-10-06T01:41:05.069709+0000","last_became_peered":"2023-10-06T01:41:05.069709+0000","last_unstale":"2023-10-06T01:41:05.074693+0000","last_undegraded":"2023-10-06T01:41:05.074693+0000","last_fullsized":"2023-10-06T01:41:05.074693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:44:50.555907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062679+0000","last_change":"2023-10-06T01:40:42.480027+0000","last_active":"2023-10-06T01:41:05.062679+0000","last_peered":"2023-10-06T01:41:05.062679+0000","last_clean":"2023-10-06T01:41:05.062679+0000","last_became_active":"2023-10-06T01:40:42.479764+0000","last_became_peered":"2023-10-06T01:40:42.479764+0000","last_unstale":"2023-10-06T01:41:05.062679+0000","last_undegraded":"2023-10-06T01:41:05.062679+0000","last_fullsized":"2023-10-06T01:41:05.062679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:40.955536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066603+0000","last_change":"2023-10-06T01:40:41.473889+0000","last_active":"2023-10-06T01:41:05.066603+0000","last_peered":"2023-10-06T01:41:05.066603+0000","last_clean":"2023-10-06T01:41:05.066603+0000","last_became_active":"2023-10-06T01:40:41.473605+0000","last_became_peered":"2023-10-06T01:40:41.473605+0000","last_unstale":"2023-10-06T01:41:05.066603+0000","last_undegraded":"2023-10-06T01:41:05.066603+0000","last_fullsized":"2023-10-06T01:41:05.066603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:38.220377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077363+0000","last_change":"2023-10-06T01:41:05.071858+0000","last_active":"2023-10-06T01:41:05.077363+0000","last_peered":"2023-10-06T01:41:05.077363+0000","last_clean":"2023-10-06T01:41:05.077363+0000","last_became_active":"2023-10-06T01:41:05.071590+0000","last_became_peered":"2023-10-06T01:41:05.071590+0000","last_unstale":"2023-10-06T01:41:05.077363+0000","last_undegraded":"2023-10-06T01:41:05.077363+0000","last_fullsized":"2023-10-06T01:41:05.077363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:39.551110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"36'9","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.059831+0000","last_change":"2023-10-06T01:40:42.471534+0000","last_active":"2023-10-06T01:41:05.059831+0000","last_peered":"2023-10-06T01:41:05.059831+0000","last_clean":"2023-10-06T01:41:05.059831+0000","last_became_active":"2023-10-06T01:40:42.471241+0000","last_became_peered":"2023-10-06T01:40:42.471241+0000","last_unstale":"2023-10-06T01:41:05.059831+0000","last_undegraded":"2023-10-06T01:41:05.059831+0000","last_fullsized":"2023-10-06T01:41:05.059831+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:51.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063540+0000","last_change":"2023-10-06T01:40:41.471125+0000","last_active":"2023-10-06T01:41:05.063540+0000","last_peered":"2023-10-06T01:41:05.063540+0000","last_clean":"2023-10-06T01:41:05.063540+0000","last_became_active":"2023-10-06T01:40:41.470901+0000","last_became_peered":"2023-10-06T01:40:41.470901+0000","last_unstale":"2023-10-06T01:41:05.063540+0000","last_undegraded":"2023-10-06T01:41:05.063540+0000","last_fullsized":"2023-10-06T01:41:05.063540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069555+0000","last_change":"2023-10-06T01:41:05.068429+0000","last_active":"2023-10-06T01:41:05.069555+0000","last_peered":"2023-10-06T01:41:05.069555+0000","last_clean":"2023-10-06T01:41:05.069555+0000","last_became_active":"2023-10-06T01:41:05.068180+0000","last_became_peered":"2023-10-06T01:41:05.068180+0000","last_unstale":"2023-10-06T01:41:05.069555+0000","last_undegraded":"2023-10-06T01:41:05.069555+0000","last_fullsized":"2023-10-06T01:41:05.069555+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:09.785044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"36'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063653+0000","last_change":"2023-10-06T01:40:42.478522+0000","last_active":"2023-10-06T01:41:05.063653+0000","last_peered":"2023-10-06T01:41:05.063653+0000","last_clean":"2023-10-06T01:41:05.063653+0000","last_became_active":"2023-10-06T01:40:42.478232+0000","last_became_peered":"2023-10-06T01:40:42.478232+0000","last_unstale":"2023-10-06T01:41:05.063653+0000","last_undegraded":"2023-10-06T01:41:05.063653+0000","last_fullsized":"2023-10-06T01:41:05.063653+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:47:46.462598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067505+0000","last_change":"2023-10-06T01:40:41.466652+0000","last_active":"2023-10-06T01:41:05.067505+0000","last_peered":"2023-10-06T01:41:05.067505+0000","last_clean":"2023-10-06T01:41:05.067505+0000","last_became_active":"2023-10-06T01:40:41.466381+0000","last_became_peered":"2023-10-06T01:40:41.466381+0000","last_unstale":"2023-10-06T01:41:05.067505+0000","last_undegraded":"2023-10-06T01:41:05.067505+0000","last_fullsized":"2023-10-06T01:41:05.067505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:19.625836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077459+0000","last_change":"2023-10-06T01:41:05.071544+0000","last_active":"2023-10-06T01:41:05.077459+0000","last_peered":"2023-10-06T01:41:05.077459+0000","last_clean":"2023-10-06T01:41:05.077459+0000","last_became_active":"2023-10-06T01:41:05.071300+0000","last_became_peered":"2023-10-06T01:41:05.071300+0000","last_unstale":"2023-10-06T01:41:05.077459+0000","last_undegraded":"2023-10-06T01:41:05.077459+0000","last_fullsized":"2023-10-06T01:41:05.077459+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"36'7","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067681+0000","last_change":"2023-10-06T01:40:42.476360+0000","last_active":"2023-10-06T01:41:05.067681+0000","last_peered":"2023-10-06T01:41:05.067681+0000","last_clean":"2023-10-06T01:41:05.067681+0000","last_became_active":"2023-10-06T01:40:42.475813+0000","last_became_peered":"2023-10-06T01:40:42.475813+0000","last_unstale":"2023-10-06T01:41:05.067681+0000","last_undegraded":"2023-10-06T01:41:05.067681+0000","last_fullsized":"2023-10-06T01:41:05.067681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:20.492529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.063996+0000","last_change":"2023-10-06T01:40:41.464545+0000","last_active":"2023-10-06T01:41:05.063996+0000","last_peered":"2023-10-06T01:41:05.063996+0000","last_clean":"2023-10-06T01:41:05.063996+0000","last_became_active":"2023-10-06T01:40:41.464275+0000","last_became_peered":"2023-10-06T01:40:41.464275+0000","last_unstale":"2023-10-06T01:41:05.063996+0000","last_undegraded":"2023-10-06T01:41:05.063996+0000","last_fullsized":"2023-10-06T01:41:05.063996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:08:56.112258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062842+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:05.062842+0000","last_peered":"2023-10-06T01:41:05.062842+0000","last_clean":"2023-10-06T01:41:05.062842+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:05.062842+0000","last_undegraded":"2023-10-06T01:41:05.062842+0000","last_fullsized":"2023-10-06T01:41:05.062842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074573+0000","last_change":"2023-10-06T01:41:05.070827+0000","last_active":"2023-10-06T01:41:05.074573+0000","last_peered":"2023-10-06T01:41:05.074573+0000","last_clean":"2023-10-06T01:41:05.074573+0000","last_became_active":"2023-10-06T01:41:05.070649+0000","last_became_peered":"2023-10-06T01:41:05.070649+0000","last_unstale":"2023-10-06T01:41:05.074573+0000","last_undegraded":"2023-10-06T01:41:05.074573+0000","last_fullsized":"2023-10-06T01:41:05.074573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:26.468333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"36'8","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067196+0000","last_change":"2023-10-06T01:40:42.478616+0000","last_active":"2023-10-06T01:41:05.067196+0000","last_peered":"2023-10-06T01:41:05.067196+0000","last_clean":"2023-10-06T01:41:05.067196+0000","last_became_active":"2023-10-06T01:40:42.478372+0000","last_became_peered":"2023-10-06T01:40:42.478372+0000","last_unstale":"2023-10-06T01:41:05.067196+0000","last_undegraded":"2023-10-06T01:41:05.067196+0000","last_fullsized":"2023-10-06T01:41:05.067196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067890+0000","last_change":"2023-10-06T01:40:41.472869+0000","last_active":"2023-10-06T01:41:05.067890+0000","last_peered":"2023-10-06T01:41:05.067890+0000","last_clean":"2023-10-06T01:41:05.067890+0000","last_became_active":"2023-10-06T01:40:41.472643+0000","last_became_peered":"2023-10-06T01:40:41.472643+0000","last_unstale":"2023-10-06T01:41:05.067890+0000","last_undegraded":"2023-10-06T01:41:05.067890+0000","last_fullsized":"2023-10-06T01:41:05.067890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067611+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:05.067611+0000","last_peered":"2023-10-06T01:41:05.067611+0000","last_clean":"2023-10-06T01:41:05.067611+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:05.067611+0000","last_undegraded":"2023-10-06T01:41:05.067611+0000","last_fullsized":"2023-10-06T01:41:05.067611+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.071384+0000","last_change":"2023-10-06T01:41:05.068635+0000","last_active":"2023-10-06T01:41:05.071384+0000","last_peered":"2023-10-06T01:41:05.071384+0000","last_clean":"2023-10-06T01:41:05.071384+0000","last_became_active":"2023-10-06T01:41:05.068364+0000","last_became_peered":"2023-10-06T01:41:05.068364+0000","last_unstale":"2023-10-06T01:41:05.071384+0000","last_undegraded":"2023-10-06T01:41:05.071384+0000","last_fullsized":"2023-10-06T01:41:05.071384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:53.027266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"36'5","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069163+0000","last_change":"2023-10-06T01:40:42.475832+0000","last_active":"2023-10-06T01:41:05.069163+0000","last_peered":"2023-10-06T01:41:05.069163+0000","last_clean":"2023-10-06T01:41:05.069163+0000","last_became_active":"2023-10-06T01:40:42.475571+0000","last_became_peered":"2023-10-06T01:40:42.475571+0000","last_unstale":"2023-10-06T01:41:05.069163+0000","last_undegraded":"2023-10-06T01:41:05.069163+0000","last_fullsized":"2023-10-06T01:41:05.069163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:52.518905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062581+0000","last_change":"2023-10-06T01:40:41.470594+0000","last_active":"2023-10-06T01:41:05.062581+0000","last_peered":"2023-10-06T01:41:05.062581+0000","last_clean":"2023-10-06T01:41:05.062581+0000","last_became_active":"2023-10-06T01:40:41.470091+0000","last_became_peered":"2023-10-06T01:40:41.470091+0000","last_unstale":"2023-10-06T01:41:05.062581+0000","last_undegraded":"2023-10-06T01:41:05.062581+0000","last_fullsized":"2023-10-06T01:41:05.062581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:50.429041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062760+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:05.062760+0000","last_peered":"2023-10-06T01:41:05.062760+0000","last_clean":"2023-10-06T01:41:05.062760+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:05.062760+0000","last_undegraded":"2023-10-06T01:41:05.062760+0000","last_fullsized":"2023-10-06T01:41:05.062760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067978+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:05.067978+0000","last_peered":"2023-10-06T01:41:05.067978+0000","last_clean":"2023-10-06T01:41:05.067978+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:05.067978+0000","last_undegraded":"2023-10-06T01:41:05.067978+0000","last_fullsized":"2023-10-06T01:41:05.067978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075630+0000","last_change":"2023-10-06T01:41:05.072823+0000","last_active":"2023-10-06T01:41:05.075630+0000","last_peered":"2023-10-06T01:41:05.075630+0000","last_clean":"2023-10-06T01:41:05.075630+0000","last_became_active":"2023-10-06T01:41:05.072511+0000","last_became_peered":"2023-10-06T01:41:05.072511+0000","last_unstale":"2023-10-06T01:41:05.075630+0000","last_undegraded":"2023-10-06T01:41:05.075630+0000","last_fullsized":"2023-10-06T01:41:05.075630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:32.843095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"36'9","reported_seq":24,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064300+0000","last_change":"2023-10-06T01:40:42.479190+0000","last_active":"2023-10-06T01:41:05.064300+0000","last_peered":"2023-10-06T01:41:05.064300+0000","last_clean":"2023-10-06T01:41:05.064300+0000","last_became_active":"2023-10-06T01:40:42.478794+0000","last_became_peered":"2023-10-06T01:40:42.478794+0000","last_unstale":"2023-10-06T01:41:05.064300+0000","last_undegraded":"2023-10-06T01:41:05.064300+0000","last_fullsized":"2023-10-06T01:41:05.064300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:00:10.270300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.069409+0000","last_change":"2023-10-06T01:40:41.472702+0000","last_active":"2023-10-06T01:41:05.069409+0000","last_peered":"2023-10-06T01:41:05.069409+0000","last_clean":"2023-10-06T01:41:05.069409+0000","last_became_active":"2023-10-06T01:40:41.472428+0000","last_became_peered":"2023-10-06T01:40:41.472428+0000","last_unstale":"2023-10-06T01:41:05.069409+0000","last_undegraded":"2023-10-06T01:41:05.069409+0000","last_fullsized":"2023-10-06T01:41:05.069409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:50.665530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067997+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:05.067997+0000","last_peered":"2023-10-06T01:41:05.067997+0000","last_clean":"2023-10-06T01:41:05.067997+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:05.067997+0000","last_undegraded":"2023-10-06T01:41:05.067997+0000","last_fullsized":"2023-10-06T01:41:05.067997+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.075181+0000","last_change":"2023-10-06T01:41:05.072739+0000","last_active":"2023-10-06T01:41:05.075181+0000","last_peered":"2023-10-06T01:41:05.075181+0000","last_clean":"2023-10-06T01:41:05.075181+0000","last_became_active":"2023-10-06T01:41:05.072486+0000","last_became_peered":"2023-10-06T01:41:05.072486+0000","last_unstale":"2023-10-06T01:41:05.075181+0000","last_undegraded":"2023-10-06T01:41:05.075181+0000","last_fullsized":"2023-10-06T01:41:05.075181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:44.012384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"36'7","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067237+0000","last_change":"2023-10-06T01:40:42.480021+0000","last_active":"2023-10-06T01:41:05.067237+0000","last_peered":"2023-10-06T01:41:05.067237+0000","last_clean":"2023-10-06T01:41:05.067237+0000","last_became_active":"2023-10-06T01:40:42.479462+0000","last_became_peered":"2023-10-06T01:40:42.479462+0000","last_unstale":"2023-10-06T01:41:05.067237+0000","last_undegraded":"2023-10-06T01:41:05.067237+0000","last_fullsized":"2023-10-06T01:41:05.067237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:26.263061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'3","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068333+0000","last_change":"2023-10-06T01:40:41.468486+0000","last_active":"2023-10-06T01:41:05.068333+0000","last_peered":"2023-10-06T01:41:05.068333+0000","last_clean":"2023-10-06T01:41:05.068333+0000","last_became_active":"2023-10-06T01:40:41.468188+0000","last_became_peered":"2023-10-06T01:40:41.468188+0000","last_unstale":"2023-10-06T01:41:05.068333+0000","last_undegraded":"2023-10-06T01:41:05.068333+0000","last_fullsized":"2023-10-06T01:41:05.068333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:21.148698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.067888+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:05.067888+0000","last_peered":"2023-10-06T01:41:05.067888+0000","last_clean":"2023-10-06T01:41:05.067888+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:05.067888+0000","last_undegraded":"2023-10-06T01:41:05.067888+0000","last_fullsized":"2023-10-06T01:41:05.067888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.074585+0000","last_change":"2023-10-06T01:41:05.071521+0000","last_active":"2023-10-06T01:41:05.074585+0000","last_peered":"2023-10-06T01:41:05.074585+0000","last_clean":"2023-10-06T01:41:05.074585+0000","last_became_active":"2023-10-06T01:41:05.071272+0000","last_became_peered":"2023-10-06T01:41:05.071272+0000","last_unstale":"2023-10-06T01:41:05.074585+0000","last_undegraded":"2023-10-06T01:41:05.074585+0000","last_fullsized":"2023-10-06T01:41:05.074585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.431329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"36'8","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062013+0000","last_change":"2023-10-06T01:40:42.485111+0000","last_active":"2023-10-06T01:41:05.062013+0000","last_peered":"2023-10-06T01:41:05.062013+0000","last_clean":"2023-10-06T01:41:05.062013+0000","last_became_active":"2023-10-06T01:40:42.484852+0000","last_became_peered":"2023-10-06T01:40:42.484852+0000","last_unstale":"2023-10-06T01:41:05.062013+0000","last_undegraded":"2023-10-06T01:41:05.062013+0000","last_fullsized":"2023-10-06T01:41:05.062013+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:01.313376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'3","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.068093+0000","last_change":"2023-10-06T01:40:41.462919+0000","last_active":"2023-10-06T01:41:05.068093+0000","last_peered":"2023-10-06T01:41:05.068093+0000","last_clean":"2023-10-06T01:41:05.068093+0000","last_became_active":"2023-10-06T01:40:41.462691+0000","last_became_peered":"2023-10-06T01:40:41.462691+0000","last_unstale":"2023-10-06T01:41:05.068093+0000","last_undegraded":"2023-10-06T01:41:05.068093+0000","last_fullsized":"2023-10-06T01:41:05.068093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066856+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:05.066856+0000","last_peered":"2023-10-06T01:41:05.066856+0000","last_clean":"2023-10-06T01:41:05.066856+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:05.066856+0000","last_undegraded":"2023-10-06T01:41:05.066856+0000","last_fullsized":"2023-10-06T01:41:05.066856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.077722+0000","last_change":"2023-10-06T01:41:05.073694+0000","last_active":"2023-10-06T01:41:05.077722+0000","last_peered":"2023-10-06T01:41:05.077722+0000","last_clean":"2023-10-06T01:41:05.077722+0000","last_became_active":"2023-10-06T01:41:05.073261+0000","last_became_peered":"2023-10-06T01:41:05.073261+0000","last_unstale":"2023-10-06T01:41:05.077722+0000","last_undegraded":"2023-10-06T01:41:05.077722+0000","last_fullsized":"2023-10-06T01:41:05.077722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:54.384575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"36'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.066331+0000","last_change":"2023-10-06T01:40:42.482655+0000","last_active":"2023-10-06T01:41:05.066331+0000","last_peered":"2023-10-06T01:41:05.066331+0000","last_clean":"2023-10-06T01:41:05.066331+0000","last_became_active":"2023-10-06T01:40:42.481980+0000","last_became_peered":"2023-10-06T01:40:42.481980+0000","last_unstale":"2023-10-06T01:41:05.066331+0000","last_undegraded":"2023-10-06T01:41:05.066331+0000","last_fullsized":"2023-10-06T01:41:05.066331+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:11.464850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062956+0000","last_change":"2023-10-06T01:40:41.461788+0000","last_active":"2023-10-06T01:41:05.062956+0000","last_peered":"2023-10-06T01:41:05.062956+0000","last_clean":"2023-10-06T01:41:05.062956+0000","last_became_active":"2023-10-06T01:40:41.461528+0000","last_became_peered":"2023-10-06T01:40:41.461528+0000","last_unstale":"2023-10-06T01:41:05.062956+0000","last_undegraded":"2023-10-06T01:41:05.062956+0000","last_fullsized":"2023-10-06T01:41:05.062956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:02.453834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062529+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:05.062529+0000","last_peered":"2023-10-06T01:41:05.062529+0000","last_clean":"2023-10-06T01:41:05.062529+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:05.062529+0000","last_undegraded":"2023-10-06T01:41:05.062529+0000","last_fullsized":"2023-10-06T01:41:05.062529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.072041+0000","last_change":"2023-10-06T01:41:05.070184+0000","last_active":"2023-10-06T01:41:05.072041+0000","last_peered":"2023-10-06T01:41:05.072041+0000","last_clean":"2023-10-06T01:41:05.072041+0000","last_became_active":"2023-10-06T01:41:05.069937+0000","last_became_peered":"2023-10-06T01:41:05.069937+0000","last_unstale":"2023-10-06T01:41:05.072041+0000","last_undegraded":"2023-10-06T01:41:05.072041+0000","last_fullsized":"2023-10-06T01:41:05.072041+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:04.051667+0000","last_clean_scrub_stamp":"2023-10-06T01:41:04.051667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:34.345292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"36'5","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062666+0000","last_change":"2023-10-06T01:40:42.470701+0000","last_active":"2023-10-06T01:41:05.062666+0000","last_peered":"2023-10-06T01:41:05.062666+0000","last_clean":"2023-10-06T01:41:05.062666+0000","last_became_active":"2023-10-06T01:40:42.470388+0000","last_became_peered":"2023-10-06T01:40:42.470388+0000","last_unstale":"2023-10-06T01:41:05.062666+0000","last_undegraded":"2023-10-06T01:41:05.062666+0000","last_fullsized":"2023-10-06T01:41:05.062666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:41.447997+0000","last_clean_scrub_stamp":"2023-10-06T01:40:41.447997+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:11.227363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.064473+0000","last_change":"2023-10-06T01:40:41.469299+0000","last_active":"2023-10-06T01:41:05.064473+0000","last_peered":"2023-10-06T01:41:05.064473+0000","last_clean":"2023-10-06T01:41:05.064473+0000","last_became_active":"2023-10-06T01:40:41.468802+0000","last_became_peered":"2023-10-06T01:40:41.468802+0000","last_unstale":"2023-10-06T01:41:05.064473+0000","last_undegraded":"2023-10-06T01:41:05.064473+0000","last_fullsized":"2023-10-06T01:41:05.064473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:40:40.443301+0000","last_clean_scrub_stamp":"2023-10-06T01:40:40.443301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:39.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T01:41:05.062127+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:05.062127+0000","last_peered":"2023-10-06T01:41:05.062127+0000","last_clean":"2023-10-06T01:41:05.062127+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:05.062127+0000","last_undegraded":"2023-10-06T01:41:05.062127+0000","last_fullsized":"2023-10-06T01:41:05.062127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.622}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.623}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":829469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.35099999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":4706304,"data_stored":4433405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.437}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":237748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.37}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.42399999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.434}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.495},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":239101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.318},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.248},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.624},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299836,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072004,"statfs":{"total":96636764160,"available":96329732096,"internally_reserved":0,"allocated":2609152,"data_stored":2336253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.436}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:13.984 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T01:41:13.984 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -s 2023-10-06T01:41:14.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.127+0000 7f7d3aa46700 1 Processor -- start 2023-10-06T01:41:14.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.127+0000 7f7d3aa46700 1 -- start start 2023-10-06T01:41:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d3aa46700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7d340c5230 0x7f7d340ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d3aa46700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3406f800 con 0x7f7d340cf030 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d3aa46700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340cd760 con 0x7f7d340cd320 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d3aa46700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3415dc90 con 0x7f7d340c5230 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d33fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7d340cf030 0x7f7d3415d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52348/0 (socket says 172.21.15.98:52348) 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d33fff700 1 -- 172.21.15.98:0/3324684093 learned_addr learned my addr 172.21.15.98:0/3324684093 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954530458 0 0) 0x7f7d3406f800 con 0x7f7d340cf030 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c003570 con 0x7f7d340cf030 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.128+0000 7f7d337fe700 1 --2- 172.21.15.98:0/3324684093 >> v2:172.21.15.184:3300/0 conn(0x7f7d340c5230 0x7f7d340ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568310490 0 0) 0x7f7d1c003570 con 0x7f7d340cf030 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 >> v2:172.21.15.184:3300/0 conn(0x7f7d340c5230 msgr2=0x7f7d340ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 --2- 172.21.15.98:0/3324684093 >> v2:172.21.15.184:3300/0 conn(0x7f7d340c5230 0x7f7d340ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 >> v1:172.21.15.184:6789/0 conn(0x7f7d340cd320 legacy=0x7f7d34159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3415df70 con 0x7f7d340cf030 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f7d240015f0 con 0x7f7d340cf030 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 >> v1:172.21.15.98:6789/0 conn(0x7f7d340cf030 legacy=0x7f7d3415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1c003a60 0x7f7d1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d1c0051c0 0x7f7d1c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3415dc90 con 0x7f7d1c004810 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c00c0d0 con 0x7f7d1c003a60 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c00c290 con 0x7f7d1c0051c0 2023-10-06T01:41:14.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d33fff700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1c003a60 0x7f7d1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d38fe3700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d1c0051c0 0x7f7d1c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 -- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d1c0051c0 msgr2=0x7f7d1c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d1c0051c0 0x7f7d1c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 -- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1c003a60 msgr2=0x7f7d1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1c003a60 0x7f7d1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d38fe3700 1 -- 172.21.15.98:0/3324684093 reap_dead start 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 -- 172.21.15.98:0/3324684093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c003570 con 0x7f7d1c004810 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d33fff700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1c003a60 0x7f7d1c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d337fe700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d1c004c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7d340c5650 tx=0x7f7d28009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85162cc0d1708c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.129+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28005a00 con 0x7f7d1c004810 2023-10-06T01:41:14.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28005b60 con 0x7f7d1c004810 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d32ffd700 1 -- 172.21.15.98:0/3324684093 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800e640 con 0x7f7d1c004810 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 -- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 msgr2=0x7f7d1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d1c004c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7d340c5650 tx=0x7f7d28009120 comp rx=0 tx=0).stop 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 -- 172.21.15.98:0/3324684093 shutdown_connections 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/3324684093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d1c004c80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 -- 172.21.15.98:0/3324684093 >> 172.21.15.98:0/3324684093 conn(0x7f7d34089e20 msgr2=0x7f7d34099740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 -- 172.21.15.98:0/3324684093 shutdown_connections 2023-10-06T01:41:14.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.130+0000 7f7d3aa46700 1 -- 172.21.15.98:0/3324684093 wait complete. 2023-10-06T01:41:14.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.131+0000 7f7d3aa46700 1 Processor -- start 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 -- start start 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d34157390 0x7f7d34156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d340b5610 0x7f7d340b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d341570f0 con 0x7f7d1c004810 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340ae6e0 con 0x7f7d340b5610 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d3aa46700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340b41f0 con 0x7f7d34157390 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d33fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35790/0 (socket says 172.21.15.98:35790) 2023-10-06T01:41:14.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.132+0000 7f7d33fff700 1 -- 172.21.15.98:0/473955985 learned_addr learned my addr 172.21.15.98:0/473955985 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 -- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d34157390 msgr2=0x7f7d34156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d34157390 0x7f7d34156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 -- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d340b5610 msgr2=0x7f7d340b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d340b5610 0x7f7d340b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 -- 172.21.15.98:0/473955985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3416a6d0 con 0x7f7d1c004810 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d33fff700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400ebc0 tx=0x7f7d24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240072e0 con 0x7f7d1c004810 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24007440 con 0x7f7d1c004810 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d340b3f10 con 0x7f7d1c004810 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240057f0 con 0x7f7d1c004810 2023-10-06T01:41:14.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.133+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d340ae370 con 0x7f7d1c004810 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.134+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3405f270 con 0x7f7d1c004810 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.134+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d24005b60 con 0x7f7d1c004810 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.135+0000 7f7d1bfff700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.135+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f7d24084b00 con 0x7f7d1c004810 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.135+0000 7f7d337fe700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.135+0000 7f7d337fe700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cf850 tx=0x7f7d28019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.139+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d240518b0 con 0x7f7d1c004810 2023-10-06T01:41:14.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.397+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7d340cb450 con 0x7f7d1c004810 2023-10-06T01:41:14.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.398+0000 7f7d1bfff700 1 -- 172.21.15.98:0/473955985 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+551 (secure 0 0 0) 0x7f7d24054ed0 con 0x7f7d1c004810 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: cluster: 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: id: c6dd341a-820f-4e37-8897-3250639f93a1 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: health: HEALTH_ERR 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: 2 filesystems are degraded 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: 2 filesystems are offline 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.400 INFO:teuthology.orchestra.run.smithi098.stdout: services: 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: mon: 3 daemons, quorum a,b,c (age 108s) 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: mgr: y(active, since 102s), standbys: x 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: mds: 0/2 daemons up (2 failed), 4 standby 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: osd: 8 osds: 8 up (since 97s), 8 in (since 105s) 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: data: 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: volumes: 0/2 healthy, 2 failed 2023-10-06T01:41:14.401 INFO:teuthology.orchestra.run.smithi098.stdout: pools: 5 pools, 137 pgs 2023-10-06T01:41:14.402 INFO:teuthology.orchestra.run.smithi098.stdout: objects: 4 objects, 6.6 MiB 2023-10-06T01:41:14.402 INFO:teuthology.orchestra.run.smithi098.stdout: usage: 2.3 GiB used, 718 GiB / 720 GiB avail 2023-10-06T01:41:14.402 INFO:teuthology.orchestra.run.smithi098.stdout: pgs: 137 active+clean 2023-10-06T01:41:14.402 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 msgr2=0x7f7d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cf850 tx=0x7f7d28019000 comp rx=0 tx=0).stop 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 msgr2=0x7f7d340aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400ebc0 tx=0x7f7d24009040 comp rx=0 tx=0).stop 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 shutdown_connections 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d340b5610 0x7f7d340b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d1c004810 0x7f7d340aee00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 --2- 172.21.15.98:0/473955985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d34157390 0x7f7d34156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 >> 172.21.15.98:0/473955985 conn(0x7f7d34089e20 msgr2=0x7f7d340685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 shutdown_connections 2023-10-06T01:41:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.400+0000 7f7d3aa46700 1 -- 172.21.15.98:0/473955985 wait complete. 2023-10-06T01:41:14.413 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init --force-init --filesystem recovery_fs --alternate-pool recovery_fs_metadata 2023-10-06T01:41:14.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.450+0000 7f1973249740 1 -- start start 2023-10-06T01:41:14.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.450+0000 7f1973249740 1 Processor -- start 2023-10-06T01:41:14.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561742819c00 0x561743451600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561742819800 0x561743450580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.459 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e3cc0 con 0x561742819000 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e2000 con 0x561742819800 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617426e23c0 con 0x561742819c00 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35806/0 (socket says 172.21.15.98:35806) 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 -- 172.21.15.98:0/1972588662 learned_addr learned my addr 172.21.15.98:0/1972588662 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196ccb8700 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561742819800 0x561743450580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196bcb6700 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561742819c00 0x561743451600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 -- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561742819c00 msgr2=0x561743451600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561742819c00 0x561743451600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 -- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561742819800 msgr2=0x561743450580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561742819800 0x561743450580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.460 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 -- 172.21.15.98:0/1972588662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617426c7ba0 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f196c4b7700 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56174346c810 tx=0x56174281a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605ab279438d2d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1969cb2700 1 -- 172.21.15.98:0/1972588662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426daf20 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1969cb2700 1 -- 172.21.15.98:0/1972588662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617426f8900 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1969cb2700 1 -- 172.21.15.98:0/1972588662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426db080 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.451+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617426c7a00 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5617426c7860 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 4 waiting for MDS map... 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1969cb2700 1 -- 172.21.15.98:0/1972588662 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x5617426e6c40 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1969cb2700 1 -- 172.21.15.98:0/1972588662 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x5617434a1200 con 0x561742819000 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 4 Got MDS map 23 2023-10-06T01:41:14.461 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 4 datascan.main: Using metadata pool output 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 4 datascan.main: connecting to RADOS... 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 1 Processor -- start 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.452+0000 7f1973249740 1 -- start start 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 0x561743452100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e2dc0 con 0x561743500800 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e3e00 con 0x561743500000 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617426e3540 con 0x561743500400 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35812/0 (socket says 172.21.15.98:35812) 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 -- 172.21.15.98:0/1734530711 learned_addr learned my addr 172.21.15.98:0/1734530711 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196c4b7700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196ccb8700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 0x561743452100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 -- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 msgr2=0x561743452680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.463 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 -- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 msgr2=0x561743452100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 0x561743452100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 -- 172.21.15.98:0/1734530711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617426c6b60 con 0x561743500800 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196ccb8700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 0x561743452100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196c4b7700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.453+0000 7f196bcb6700 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x56174346d830 tx=0x5617434aec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8e024f803ece41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1967cae700 1 -- 172.21.15.98:0/1734530711 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426daf20 con 0x561743500800 2023-10-06T01:41:14.464 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1967cae700 1 -- 172.21.15.98:0/1734530711 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617426f8300 con 0x561743500800 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1967cae700 1 -- 172.21.15.98:0/1734530711 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426db080 con 0x561743500800 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 -- 172.21.15.98:0/1734530711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 msgr2=0x561743452c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x56174346d830 tx=0x5617434aec00 comp rx=0 tx=0).stop 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 -- 172.21.15.98:0/1734530711 shutdown_connections 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500800 0x561743452c00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500000 0x561743452100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.465 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 --2- 172.21.15.98:0/1734530711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.468 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 -- 172.21.15.98:0/1734530711 >> 172.21.15.98:0/1734530711 conn(0x561742818c00 msgr2=0x56174342c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 -- 172.21.15.98:0/1734530711 shutdown_connections 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.454+0000 7f1973249740 1 -- 172.21.15.98:0/1734530711 wait complete. 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 Processor -- start 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 -- start start 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500800 0x561743453180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e23c0 con 0x561743500000 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617426e3180 con 0x561743500800 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f1973249740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617426e3040 con 0x561743500400 2023-10-06T01:41:14.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196ccb8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196ccb8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35820/0 (socket says 172.21.15.98:35820) 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 learned_addr learned my addr 172.21.15.98:0/973020150 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196c4b7700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196bcb6700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500800 0x561743453180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 msgr2=0x561743452c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.455+0000 7f196ccb8700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561743500400 0x561743452c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500800 msgr2=0x561743453180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f196ccb8700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561743500800 0x561743453180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617426c7ba0 con 0x561743500000 2023-10-06T01:41:14.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f196ccb8700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56174346c750 tx=0x5617434aed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f19664ab700 1 -- 172.21.15.98:0/973020150 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426daf20 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f19664ab700 1 -- 172.21.15.98:0/973020150 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617426f8c00 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5617426c6000 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f19664ab700 1 -- 172.21.15.98:0/973020150 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617426db080 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617426c69c0 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.456+0000 7f1973249740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f19664ab700 1 -- 172.21.15.98:0/973020150 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x561743418b00 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f19664ab700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 0x561743453700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f19664ab700 1 -- 172.21.15.98:0/973020150 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x5617426e6a80 con 0x561743500000 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f196c4b7700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 0x561743453700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x561743500c00 0x561743602000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.471 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.457+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x561743501000 con 0x561743500c00 2023-10-06T01:41:14.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.458+0000 7f196c4b7700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 0x561743453700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56174346d230 tx=0x56174281a0c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.472 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.458+0000 7f196bcb6700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x561743500c00 0x561743602000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.475 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.458+0000 7f196bcb6700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x561743500c00 0x561743602000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.458+0000 7f196bcb6700 1 -- 172.21.15.98:0/973020150 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x561743484fc0 con 0x561743500c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.458+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:2 8.14 8:287034c4:::1.00000000.inode:head [writefull 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x561743501800 con 0x561743500c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.460+0000 7f196bcb6700 1 -- 172.21.15.98:0/973020150 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~554] v44'1 uv1 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x561743484fc0 con 0x561743500c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.460+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x561743501c00 0x561743602580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.460+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:3 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x561743501000 con 0x561743501c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.460+0000 7f196ccb8700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x561743501c00 0x561743602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.460+0000 7f196ccb8700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x561743501c00 0x561743602580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.461+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x561743485680 con 0x561743501c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.461+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:4 8.1f 8:fe07bba1:::100.00000000.inode:head [writefull 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x561743501800 con 0x561743501c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.462+0000 7f196ccb8700 1 -- 172.21.15.98:0/973020150 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~554] v44'1 uv1 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x561743485680 con 0x561743501c00 2023-10-06T01:41:14.476 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.462+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x561743501000 0x561743602b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.462+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:5 8.13 8:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x561743501800 con 0x561743501000 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.462+0000 7f196c4b7700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x561743501000 0x561743602b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.462+0000 7f196c4b7700 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x561743501000 0x561743602b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.463+0000 7f196c4b7700 1 -- 172.21.15.98:0/973020150 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x561743485d40 con 0x561743501000 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.463+0000 7f1973249740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.463+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:6 8.13 8:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x561742818800 con 0x561743501000 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f196c4b7700 1 -- 172.21.15.98:0/973020150 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v44'1 uv1 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x561743485d40 con 0x561743501000 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x561743501c00 msgr2=0x561743602580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x561743501c00 0x561743602580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.477 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x561743501000 msgr2=0x561743602b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x561743501000 0x561743602b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x561743500c00 msgr2=0x561743602000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x561743500c00 0x561743602000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f196bcb6700 1 -- 172.21.15.98:0/973020150 reap_dead start 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 msgr2=0x561743453700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 0x561743453700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56174346d230 tx=0x56174281a0c0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 msgr2=0x561743452680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56174346c750 tx=0x5617434aed20 comp rx=0 tx=0).stop 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 shutdown_connections 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561743500000 0x561743452680 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/973020150 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x561743501400 0x561743453700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 >> 172.21.15.98:0/973020150 conn(0x561742818c00 msgr2=0x56174342c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 shutdown_connections 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/973020150 wait complete. 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 msgr2=0x561743451b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56174346c810 tx=0x56174281a480 comp rx=0 tx=0).stop 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 shutdown_connections 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561742819800 0x561743450580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561742819000 0x561743451b80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 --2- 172.21.15.98:0/1972588662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561742819c00 0x561743451600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 >> 172.21.15.98:0/1972588662 conn(0x561742818000 msgr2=0x56174342b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 shutdown_connections 2023-10-06T01:41:14.479 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.464+0000 7f1973249740 1 -- 172.21.15.98:0/1972588662 wait complete. 2023-10-06T01:41:14.480 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init', '--force-init', '--filesystem', 'recovery_fs', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.066327, result: 2023-10-06T01:41:14.480 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset session 2023-10-06T01:41:14.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.550+0000 7f096207d240 1 -- start start 2023-10-06T01:41:14.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.550+0000 7f096207d240 1 Processor -- start 2023-10-06T01:41:14.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.550+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f000 0x55ca4c9e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4bd9e800 0x55ca4c9e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc69cc0 con 0x55ca4bd9fc00 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc69540 con 0x55ca4bd9f000 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc68000 con 0x55ca4bd9e800 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35828/0 (socket says 172.21.15.98:35828) 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 -- 172.21.15.98:0/2436666158 learned_addr learned my addr 172.21.15.98:0/2436666158 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095bd4c700 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4bd9e800 0x55ca4c9e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095b54b700 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f000 0x55ca4c9e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 -- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4bd9e800 msgr2=0x55ca4c9e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4bd9e800 0x55ca4c9e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 -- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f000 msgr2=0x55ca4c9e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f000 0x55ca4c9e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 -- 172.21.15.98:0/2436666158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4bc4da00 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f095ad4a700 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55ca4c9b5ec0 tx=0x55ca4ca22000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88cd6a3ea8fc87e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f0958d46700 1 -- 172.21.15.98:0/2436666158 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61080 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f0958d46700 1 -- 172.21.15.98:0/2436666158 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca4bc7e600 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f0958d46700 1 -- 172.21.15.98:0/2436666158 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61600 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4bc4dba0 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ca4bc4dd40 con 0x55ca4bd9fc00 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 4 waiting for MDS map... 2023-10-06T01:41:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f0958d46700 1 -- 172.21.15.98:0/2436666158 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55ca4bc6cc40 con 0x55ca4bd9fc00 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f0958d46700 1 -- 172.21.15.98:0/2436666158 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x55ca4ca20600 con 0x55ca4bd9fc00 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.551+0000 7f096207d240 4 Got MDS map 23 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.552+0000 7f096207d240 10 main: main 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.552+0000 7f096207d240 4 main: connecting to RADOS... 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.552+0000 7f096207d240 1 Processor -- start 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.552+0000 7f096207d240 1 -- start start 2023-10-06T01:41:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc683c0 con 0x55ca4bd9ec00 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc68dc0 con 0x55ca4bd9f800 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bcf0dc0 con 0x55ca4ca86000 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35832/0 (socket says 172.21.15.98:35832) 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 -- 172.21.15.98:0/1806614684 learned_addr learned my addr 172.21.15.98:0/1806614684 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095ad4a700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095bd4c700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 -- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 msgr2=0x55ca4c9e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 -- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 msgr2=0x55ca4c9e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 -- 172.21.15.98:0/1806614684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4bc4da00 con 0x55ca4bd9ec00 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095bd4c700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f095b54b700 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55ca4ca29380 tx=0x55ca4ca3ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4118949d652895c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f0956d42700 1 -- 172.21.15.98:0/1806614684 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61080 con 0x55ca4bd9ec00 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f0956d42700 1 -- 172.21.15.98:0/1806614684 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca4bc7ec00 con 0x55ca4bd9ec00 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.553+0000 7f0956d42700 1 -- 172.21.15.98:0/1806614684 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61600 con 0x55ca4bd9ec00 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- 172.21.15.98:0/1806614684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 msgr2=0x55ca4c9e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55ca4ca29380 tx=0x55ca4ca3ec00 comp rx=0 tx=0).stop 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- 172.21.15.98:0/1806614684 shutdown_connections 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 --2- 172.21.15.98:0/1806614684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6100 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- 172.21.15.98:0/1806614684 >> 172.21.15.98:0/1806614684 conn(0x55ca4bd9e400 msgr2=0x55ca4c9af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- 172.21.15.98:0/1806614684 shutdown_connections 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- 172.21.15.98:0/1806614684 wait complete. 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 Processor -- start 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.554+0000 7f096207d240 1 -- start start 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc69540 con 0x55ca4bd9ec00 2023-10-06T01:41:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc69180 con 0x55ca4bd9f800 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca4bc69040 con 0x55ca4ca86000 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35846/0 (socket says 172.21.15.98:35846) 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 -- 172.21.15.98:0/1427297872 learned_addr learned my addr 172.21.15.98:0/1427297872 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095bd4c700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095ad4a700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 -- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 msgr2=0x55ca4c9e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4ca86000 0x55ca4c9e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 -- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 msgr2=0x55ca4c9e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.565 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f800 0x55ca4c9e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 -- 172.21.15.98:0/1427297872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4bc4c9c0 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095b54b700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6680 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ca4ca94060 tx=0x55ca4ca3ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095553f700 1 -- 172.21.15.98:0/1427297872 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61080 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095553f700 1 -- 172.21.15.98:0/1427297872 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca4bc7ed80 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca4bc4dba0 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f095553f700 1 -- 172.21.15.98:0/1427297872 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca4bc61600 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4bc4c000 con 0x55ca4bd9ec00 2023-10-06T01:41:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.555+0000 7f096207d240 4 main: resolving pool 8 2023-10-06T01:41:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095553f700 1 -- 172.21.15.98:0/1427297872 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55ca4c99f080 con 0x55ca4bd9ec00 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095553f700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55ca4ca9c400 0x55ca4c9e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095553f700 1 -- 172.21.15.98:0/1427297872 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55ca4bc6ca80 con 0x55ca4bd9ec00 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f096207d240 4 main: creating IoCtx.. 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095bd4c700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55ca4ca9c400 0x55ca4c9e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f096207d240 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55ca4ca86400 0x55ca4c9e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:1 8.b 8:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55ca4ca86800 con 0x55ca4ca86400 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095ad4a700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55ca4ca86400 0x55ca4c9e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095bd4c700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55ca4ca9c400 0x55ca4c9e7700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55ca4bcb7ad0 tx=0x55ca4c9bbbc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.557+0000 7f095ad4a700 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55ca4ca86400 0x55ca4c9e7180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.561+0000 7f095ad4a700 1 -- 172.21.15.98:0/1427297872 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v44'1 uv1 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55ca4ca04fc0 con 0x55ca4ca86400 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "0": { 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "data": {}, 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "result": 0 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.561+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55ca4ca86400 msgr2=0x55ca4c9e7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55ca4ca86400 0x55ca4c9e7180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55ca4ca9c400 msgr2=0x55ca4c9e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55ca4ca9c400 0x55ca4c9e7700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55ca4bcb7ad0 tx=0x55ca4c9bbbc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 msgr2=0x55ca4c9e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/1427297872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9ec00 0x55ca4c9e6680 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ca4ca94060 tx=0x55ca4ca3ecc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f095b54b700 1 -- 172.21.15.98:0/1427297872 reap_dead start 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 shutdown_connections 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 >> 172.21.15.98:0/1427297872 conn(0x55ca4bd9e400 msgr2=0x55ca4c9af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 shutdown_connections 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/1427297872 wait complete. 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 msgr2=0x55ca4c9e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55ca4c9b5ec0 tx=0x55ca4ca22000 comp rx=0 tx=0).stop 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 shutdown_connections 2023-10-06T01:41:14.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca4bd9e800 0x55ca4c9e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca4bd9f000 0x55ca4c9e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 --2- 172.21.15.98:0/2436666158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca4bd9fc00 0x55ca4c9e4000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 >> 172.21.15.98:0/2436666158 conn(0x55ca4bd9e000 msgr2=0x55ca4c9b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 shutdown_connections 2023-10-06T01:41:14.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.562+0000 7f096207d240 1 -- 172.21.15.98:0/2436666158 wait complete. 2023-10-06T01:41:14.573 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'session'] in time 0:00:00.093403, result: { "0": { "data": {}, "result": 0 } } 2023-10-06T01:41:14.573 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset snap 2023-10-06T01:41:14.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.644+0000 7f5096496240 1 -- start start 2023-10-06T01:41:14.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.644+0000 7f5096496240 1 Processor -- start 2023-10-06T01:41:14.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 0x55d34c87eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c898c00 0x55d34c87f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.653 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07cc0 con 0x55d34c898400 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07540 con 0x55d34c898800 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07e00 con 0x55d34c898c00 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35876/0 (socket says 172.21.15.98:35876) 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 -- 172.21.15.98:0/3879934891 learned_addr learned my addr 172.21.15.98:0/3879934891 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f163700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 0x55d34c87eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5090165700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c898c00 0x55d34c87f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 -- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c898c00 msgr2=0x55d34c87f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c898c00 0x55d34c87f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.654 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 -- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 msgr2=0x55d34c87eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 0x55d34c87eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 -- 172.21.15.98:0/3879934891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34baea340 con 0x55d34c898400 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f163700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 0x55d34c87eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508f964700 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d34c853ce0 tx=0x55d34c859a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6515c1855a9bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508d15f700 1 -- 172.21.15.98:0/3879934891 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34baff080 con 0x55d34c898400 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508d15f700 1 -- 172.21.15.98:0/3879934891 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d34bb1cf00 con 0x55d34c898400 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f508d15f700 1 -- 172.21.15.98:0/3879934891 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34baff600 con 0x55d34c898400 2023-10-06T01:41:14.655 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d34baea000 con 0x55d34c898400 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d34baeaea0 con 0x55d34c898400 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.645+0000 7f5096496240 4 waiting for MDS map... 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f508d15f700 1 -- 172.21.15.98:0/3879934891 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55d34bb0ac40 con 0x55d34c898400 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f508d15f700 1 -- 172.21.15.98:0/3879934891 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x55d34c890900 con 0x55d34c898400 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f5096496240 4 Got MDS map 23 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f5096496240 10 main: main 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f5096496240 4 main: connecting to RADOS... 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f5096496240 1 Processor -- start 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.646+0000 7f5096496240 1 -- start start 2023-10-06T01:41:14.656 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c899c00 0x55d34c87fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07b80 con 0x55d34c899800 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07540 con 0x55d34c899c00 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb07cc0 con 0x55d34c90e000 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35880/0 (socket says 172.21.15.98:35880) 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 -- 172.21.15.98:0/1769417143 learned_addr learned my addr 172.21.15.98:0/1769417143 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.657 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508f964700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c899c00 0x55d34c87fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508f163700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 -- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 msgr2=0x55d34c880100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 -- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c899c00 msgr2=0x55d34c87fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c899c00 0x55d34c87fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 -- 172.21.15.98:0/1769417143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34baeb860 con 0x55d34c899800 2023-10-06T01:41:14.658 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508f163700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f5090165700 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d34c8acff0 tx=0x55d34c8bcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ef71caa6754621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508b15b700 1 -- 172.21.15.98:0/1769417143 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34baff080 con 0x55d34c899800 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508b15b700 1 -- 172.21.15.98:0/1769417143 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d34bb1c600 con 0x55d34c899800 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.647+0000 7f508b15b700 1 -- 172.21.15.98:0/1769417143 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34baff600 con 0x55d34c899800 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- 172.21.15.98:0/1769417143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 msgr2=0x55d34c87f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d34c8acff0 tx=0x55d34c8bcc00 comp rx=0 tx=0).stop 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- 172.21.15.98:0/1769417143 shutdown_connections 2023-10-06T01:41:14.659 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899800 0x55d34c87f600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 --2- 172.21.15.98:0/1769417143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c899c00 0x55d34c87fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- 172.21.15.98:0/1769417143 >> 172.21.15.98:0/1769417143 conn(0x55d34c899000 msgr2=0x55d34c84d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- 172.21.15.98:0/1769417143 shutdown_connections 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- 172.21.15.98:0/1769417143 wait complete. 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 Processor -- start 2023-10-06T01:41:14.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.648+0000 7f5096496240 1 -- start start 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 0x55d34c880c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c899800 0x55d34c880680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb06dc0 con 0x55d34c899c00 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb063c0 con 0x55d34c90e000 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5096496240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34bb06000 con 0x55d34c899800 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 0x55d34c880c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 0x55d34c880c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35884/0 (socket says 172.21.15.98:35884) 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 -- 172.21.15.98:0/2361910648 learned_addr learned my addr 172.21.15.98:0/2361910648 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f508f163700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f508f964700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c899800 0x55d34c880680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 -- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c899800 msgr2=0x55d34c880680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c899800 0x55d34c880680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 -- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c90e000 msgr2=0x55d34c880100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c90e000 0x55d34c880100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 -- 172.21.15.98:0/2361910648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34baea9c0 con 0x55d34c899c00 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f508f964700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c899800 0x55d34c880680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.649+0000 7f5090165700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 0x55d34c880c00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55d34c8adcb0 tx=0x55d34c8bccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5089958700 1 -- 172.21.15.98:0/2361910648 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34bafedc0 con 0x55d34c899c00 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5089958700 1 -- 172.21.15.98:0/2361910648 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d34bb1c000 con 0x55d34c899c00 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d34baeba00 con 0x55d34c899c00 2023-10-06T01:41:14.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5089958700 1 -- 172.21.15.98:0/2361910648 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d34baff080 con 0x55d34c899c00 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d34baea340 con 0x55d34c899c00 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.650+0000 7f5096496240 4 main: resolving pool 8 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5089958700 1 -- 172.21.15.98:0/2361910648 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55d34c83d080 con 0x55d34c899c00 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5089958700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55d34c934400 0x55d34c881700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5089958700 1 -- 172.21.15.98:0/2361910648 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55d34bb0aa80 con 0x55d34c899c00 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f508f964700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55d34c934400 0x55d34c881700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5096496240 4 main: creating IoCtx.. 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5096496240 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55d34c90e400 0x55d34c87fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:1 8.d 8:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55d34c90e800 con 0x55d34c90e400 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.651+0000 7f508f163700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55d34c90e400 0x55d34c87fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.652+0000 7f508f964700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55d34c934400 0x55d34c881700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55d34c8ac270 tx=0x55d34c858ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.652+0000 7f508f163700 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55d34c90e400 0x55d34c87fb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f508f163700 1 -- 172.21.15.98:0/2361910648 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v44'1 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55d34c8a0fc0 con 0x55d34c90e400 2023-10-06T01:41:14.664 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:14.664 INFO:teuthology.orchestra.run.smithi098.stdout: "result": 0 2023-10-06T01:41:14.664 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:14.664 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55d34c90e400 msgr2=0x55d34c87fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55d34c90e400 0x55d34c87fb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55d34c934400 msgr2=0x55d34c881700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55d34c934400 0x55d34c881700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55d34c8ac270 tx=0x55d34c858ae0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 msgr2=0x55d34c880c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 --2- 172.21.15.98:0/2361910648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c899c00 0x55d34c880c00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55d34c8adcb0 tx=0x55d34c8bccc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f508f163700 1 -- 172.21.15.98:0/2361910648 reap_dead start 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 shutdown_connections 2023-10-06T01:41:14.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 >> 172.21.15.98:0/2361910648 conn(0x55d34c899000 msgr2=0x55d34c84d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.653+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 shutdown_connections 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/2361910648 wait complete. 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 msgr2=0x55d34c87e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d34c853ce0 tx=0x55d34c859a40 comp rx=0 tx=0).stop 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 shutdown_connections 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d34c898c00 0x55d34c87f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d34c898400 0x55d34c87e580 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 --2- 172.21.15.98:0/3879934891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d34c898800 0x55d34c87eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 >> 172.21.15.98:0/3879934891 conn(0x55d34c898000 msgr2=0x55d34c84d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 shutdown_connections 2023-10-06T01:41:14.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.654+0000 7f5096496240 1 -- 172.21.15.98:0/3879934891 wait complete. 2023-10-06T01:41:14.669 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'snap'] in time 0:00:00.095216, result: { "result": 0 } 2023-10-06T01:41:14.669 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset inode 2023-10-06T01:41:14.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.740+0000 7f1f0ba98240 1 -- start start 2023-10-06T01:41:14.742 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.740+0000 7f1f0ba98240 1 Processor -- start 2023-10-06T01:41:14.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b90400 0x55a2b0b6a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 0x55a2b0b6b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.744 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3cc0 con 0x55a2b0b90800 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3540 con 0x55a2b0b90c00 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3b80 con 0x55a2b0b90400 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35900/0 (socket says 172.21.15.98:35900) 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 -- 172.21.15.98:0/3813956503 learned_addr learned my addr 172.21.15.98:0/3813956503 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04f66700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b90400 0x55a2b0b6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.745 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f05767700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 0x55a2b0b6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 -- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b90400 msgr2=0x55a2b0b6a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b90400 0x55a2b0b6a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 -- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 msgr2=0x55a2b0b6b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 0x55a2b0b6b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 -- 172.21.15.98:0/3813956503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2afdd6340 con 0x55a2b0b90800 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f05767700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 0x55a2b0b6b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.746 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f04765700 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b3fdd0 tx=0x55a2b0b301e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd15d57d74b5f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.741+0000 7f1f02761700 1 -- 172.21.15.98:0/3813956503 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeb600 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f02761700 1 -- 172.21.15.98:0/3813956503 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2afe08480 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f02761700 1 -- 172.21.15.98:0/3813956503 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeb1e0 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2afdd6000 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a2afdd6ea0 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 4 waiting for MDS map... 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f02761700 1 -- 172.21.15.98:0/3813956503 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55a2afdf6c40 con 0x55a2b0b90800 2023-10-06T01:41:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f02761700 1 -- 172.21.15.98:0/3813956503 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x55a2b0baa600 con 0x55a2b0b90800 2023-10-06T01:41:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 4 Got MDS map 23 2023-10-06T01:41:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 10 main: main 2023-10-06T01:41:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 4 main: connecting to RADOS... 2023-10-06T01:41:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 1 Processor -- start 2023-10-06T01:41:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.742+0000 7f1f0ba98240 1 -- start start 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf2dc0 con 0x55a2b0c04000 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3540 con 0x55a2b0b91c00 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf23c0 con 0x55a2b0b91800 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35904/0 (socket says 172.21.15.98:35904) 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 -- 172.21.15.98:0/1666332570 learned_addr learned my addr 172.21.15.98:0/1666332570 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f05767700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04f66700 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 -- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 msgr2=0x55a2b0b6b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 -- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 msgr2=0x55a2b0b6bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 -- 172.21.15.98:0/1666332570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2afdd7860 con 0x55a2b0c04000 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.743+0000 7f1f04765700 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b9e780 tx=0x55a2b0b30060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0b316469852235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0075d700 1 -- 172.21.15.98:0/1666332570 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeaf20 con 0x55a2b0c04000 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0075d700 1 -- 172.21.15.98:0/1666332570 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2afe08300 con 0x55a2b0c04000 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0075d700 1 -- 172.21.15.98:0/1666332570 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeb600 con 0x55a2b0c04000 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 -- 172.21.15.98:0/1666332570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 msgr2=0x55a2b0b6c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.754 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b9e780 tx=0x55a2b0b30060 comp rx=0 tx=0).stop 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 -- 172.21.15.98:0/1666332570 shutdown_connections 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c04000 0x55a2b0b6c100 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/1666332570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 -- 172.21.15.98:0/1666332570 >> 172.21.15.98:0/1666332570 conn(0x55a2b0b91000 msgr2=0x55a2b0b39800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 -- 172.21.15.98:0/1666332570 shutdown_connections 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.744+0000 7f1f0ba98240 1 -- 172.21.15.98:0/1666332570 wait complete. 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 Processor -- start 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 -- start start 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.755 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 0x55a2b0b6c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf2000 con 0x55a2b0c08000 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3180 con 0x55a2b0b91c00 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f0ba98240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afdf3040 con 0x55a2b0b91800 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 0x55a2b0b6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 0x55a2b0b6c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35910/0 (socket says 172.21.15.98:35910) 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 -- 172.21.15.98:0/2468941566 learned_addr learned my addr 172.21.15.98:0/2468941566 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f05767700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04f66700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 -- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 msgr2=0x55a2b0b6c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b91800 0x55a2b0b6c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 -- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 msgr2=0x55a2b0b6cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f04765700 1 -- 172.21.15.98:0/2468941566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2afdd6ea0 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.745+0000 7f1f05767700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b91c00 0x55a2b0b6cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1f04765700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 0x55a2b0b6c100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55a2b0c30000 tx=0x55a2b0bb6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1efef5a700 1 -- 172.21.15.98:0/2468941566 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeb1e0 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1efef5a700 1 -- 172.21.15.98:0/2468941566 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2b0bb2600 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a2afdd7a00 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1efef5a700 1 -- 172.21.15.98:0/2468941566 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a2afdeadc0 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2afdd7860 con 0x55a2b0c08000 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.746+0000 7f1f0ba98240 4 main: resolving pool 8 2023-10-06T01:41:14.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1efef5a700 1 -- 172.21.15.98:0/2468941566 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55a2b0b29080 con 0x55a2b0c08000 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1efef5a700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55a2b0c2e800 0x55a2b0b6d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1efef5a700 1 -- 172.21.15.98:0/2468941566 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55a2afdf6a80 con 0x55a2b0c08000 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1f05767700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55a2b0c2e800 0x55a2b0b6d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1f0ba98240 4 main: creating IoCtx.. 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55a2b0c08400 0x55a2b0b6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:1 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55a2b0c08800 con 0x55a2b0c08400 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.747+0000 7f1f04f66700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55a2b0c08400 0x55a2b0b6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.748+0000 7f1f05767700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55a2b0c2e800 0x55a2b0b6d700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b3e810 tx=0x55a2b0b45980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.748+0000 7f1f04f66700 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55a2b0c08400 0x55a2b0b6bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f04f66700 1 -- 172.21.15.98:0/2468941566 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v44'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55a2b0b98fc0 con 0x55a2b0c08400 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout: "0": { 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout: "data": {}, 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout: "result": 0 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:14.759 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:14.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55a2b0c08400 msgr2=0x55a2b0b6bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55a2b0c08400 0x55a2b0b6bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55a2b0c2e800 msgr2=0x55a2b0b6d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.761 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55a2b0c2e800 0x55a2b0b6d700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b3e810 tx=0x55a2b0b45980 comp rx=0 tx=0).stop 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 msgr2=0x55a2b0b6c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/2468941566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0c08000 0x55a2b0b6c100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55a2b0c30000 tx=0x55a2b0bb6cc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f04f66700 1 -- 172.21.15.98:0/2468941566 reap_dead start 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 shutdown_connections 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 >> 172.21.15.98:0/2468941566 conn(0x55a2b0b91000 msgr2=0x55a2b0b39800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 shutdown_connections 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/2468941566 wait complete. 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 msgr2=0x55a2b0b6ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55a2b0b3fdd0 tx=0x55a2b0b301e0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.762 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 shutdown_connections 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a2b0b90c00 0x55a2b0b6b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a2b0b90400 0x55a2b0b6a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 --2- 172.21.15.98:0/3813956503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2b0b90800 0x55a2b0b6ab00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.749+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 >> 172.21.15.98:0/3813956503 conn(0x55a2b0b90000 msgr2=0x55a2b0b39000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.750+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 shutdown_connections 2023-10-06T01:41:14.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.750+0000 7f1f0ba98240 1 -- 172.21.15.98:0/3813956503 wait complete. 2023-10-06T01:41:14.763 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'inode'] in time 0:00:00.094443, result: { "0": { "data": {}, "result": 0 } } 2023-10-06T01:41:14.763 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --alternate-pool recovery_fs_metadata --filesystem cephfs cephfs_data 2023-10-06T01:41:14.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.839+0000 7fe9bb343740 1 -- start start 2023-10-06T01:41:14.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.839+0000 7fe9bb343740 1 Processor -- start 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72249800 0x55eb72e8f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 0x55eb72e8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113cc0 con 0x55eb72248c00 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113540 con 0x55eb72249000 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb72112000 con 0x55eb72249800 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35922/0 (socket says 172.21.15.98:35922) 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 -- 172.21.15.98:0/3506094302 learned_addr learned my addr 172.21.15.98:0/3506094302 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 0x55eb72e8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 -- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72249800 msgr2=0x55eb72e8f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72249800 0x55eb72e8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 -- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 msgr2=0x55eb72e8e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 0x55eb72e8e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 -- 172.21.15.98:0/3506094302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb720f7860 con 0x55eb72248c00 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 0x55eb72e8e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55eb72ea2630 tx=0x55eb72eca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3744fa7fc620a4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b1dac700 1 -- 172.21.15.98:0/3506094302 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b080 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b1dac700 1 -- 172.21.15.98:0/3506094302 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb72128900 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9b1dac700 1 -- 172.21.15.98:0/3506094302 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b600 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb720f7a00 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55eb720f7ba0 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.840+0000 7fe9bb343740 4 waiting for MDS map... 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9b1dac700 1 -- 172.21.15.98:0/3506094302 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55eb72116c40 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9b1dac700 1 -- 172.21.15.98:0/3506094302 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x55eb72ec8600 con 0x55eb72248c00 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9bb343740 4 Got MDS map 23 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9bb343740 4 datascan.main: Using metadata pool output 2023-10-06T01:41:14.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9bb343740 4 datascan.main: connecting to RADOS... 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9bb343740 1 Processor -- start 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.841+0000 7fe9bb343740 1 -- start start 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72248400 0x55eb72e8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 0x55eb72e90680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb721123c0 con 0x55eb72249c00 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72112dc0 con 0x55eb72f2c000 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113540 con 0x55eb72248400 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35928/0 (socket says 172.21.15.98:35928) 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 -- 172.21.15.98:0/3477194747 learned_addr learned my addr 172.21.15.98:0/3477194747 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72248400 0x55eb72e8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 -- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72248400 msgr2=0x55eb72e8fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72248400 0x55eb72e8fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 0x55eb72e90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 -- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 msgr2=0x55eb72e90680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 0x55eb72e90680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 -- 172.21.15.98:0/3477194747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb720f7d40 con 0x55eb72249c00 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 0x55eb72e90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.842+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55eb72ea3b60 tx=0x55eb72edcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89608160184527a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9afda8700 1 -- 172.21.15.98:0/3477194747 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b080 con 0x55eb72249c00 2023-10-06T01:41:14.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9afda8700 1 -- 172.21.15.98:0/3477194747 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb72128480 con 0x55eb72249c00 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9afda8700 1 -- 172.21.15.98:0/3477194747 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b600 con 0x55eb72249c00 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 -- 172.21.15.98:0/3477194747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 msgr2=0x55eb72e90100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55eb72ea3b60 tx=0x55eb72edcc00 comp rx=0 tx=0).stop 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 -- 172.21.15.98:0/3477194747 shutdown_connections 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72f2c000 0x55eb72e90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72248400 0x55eb72e8fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3477194747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72249c00 0x55eb72e90100 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 -- 172.21.15.98:0/3477194747 >> 172.21.15.98:0/3477194747 conn(0x55eb72248800 msgr2=0x55eb72e5b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 -- 172.21.15.98:0/3477194747 shutdown_connections 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 -- 172.21.15.98:0/3477194747 wait complete. 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.843+0000 7fe9bb343740 1 Processor -- start 2023-10-06T01:41:14.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 -- start start 2023-10-06T01:41:14.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249c00 0x55eb72e91180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72f38000 0x55eb72e90680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113cc0 con 0x55eb72248400 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113180 con 0x55eb72249c00 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9bb343740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb72113040 con 0x55eb72f38000 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35942/0 (socket says 172.21.15.98:35942) 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 learned_addr learned my addr 172.21.15.98:0/1672934622 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249c00 0x55eb72e91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72f38000 0x55eb72e90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72f38000 msgr2=0x55eb72e90680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72f38000 0x55eb72e90680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249c00 msgr2=0x55eb72e91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249c00 0x55eb72e91180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb720f6000 con 0x55eb72248400 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.844+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72f38000 0x55eb72e90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55eb72ef26c0 tx=0x55eb72edccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9ae5a5700 1 -- 172.21.15.98:0/1672934622 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b1e0 con 0x55eb72248400 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9ae5a5700 1 -- 172.21.15.98:0/1672934622 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb72128300 con 0x55eb72248400 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb720f69c0 con 0x55eb72248400 2023-10-06T01:41:14.863 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9ae5a5700 1 -- 172.21.15.98:0/1672934622 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eb7210b080 con 0x55eb72248400 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb720f7860 con 0x55eb72248400 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.845+0000 7fe9bb343740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9ae5a5700 1 -- 172.21.15.98:0/1672934622 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55eb72e49080 con 0x55eb72248400 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9ae5a5700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 0x55eb7302e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9ae5a5700 1 -- 172.21.15.98:0/1672934622 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55eb72116a80 con 0x55eb72248400 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 0x55eb7302e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9bb343740 4 datascan.main: data pool 'cephfs_data' has ID 7 2023-10-06T01:41:14.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9bb343740 4 datascan.main: opening data pool 'cephfs_data' 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.846+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55eb72f38400 0x55eb72e90100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.847+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:1 7.0 7:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb72f38400 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.847+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 0x55eb7302e000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55eb72e4c870 tx=0x55eb72e65bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.847+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55eb72f38400 0x55eb72e90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.847+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55eb72f38400 0x55eb72e90100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9af5a7700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55eb72f56800 0x55eb7302e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:2 7.20 7:04000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f56c00 con 0x55eb72f56800 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55eb72f56800 0x55eb7302e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55eb72f56800 0x55eb7302e580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9afda8700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55eb72f57400 0x55eb7302eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:3 7.10 7:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57800 con 0x55eb72f57400 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.848+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55eb72f57400 0x55eb7302eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.849+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55eb72f57400 0x55eb7302eb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.849+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.849+0000 7fe9af5a7700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55eb72f57000 0x55eb7302f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.849+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:4 7.30 7:0c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056000 con 0x55eb72f57000 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.849+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55eb72f57000 0x55eb7302f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55eb72f57000 0x55eb7302f080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 1 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:5 7.8 7:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57c00 con 0x55eb72f56800 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 2 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:6 7.28 7:14000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056400 con 0x55eb72f57000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 2 ==== osd_op_reply(6 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9afda8700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55eb73056000 0x55eb7302fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.850+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:7 7.18 7:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056c00 con 0x55eb73056000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55eb73056000 0x55eb7302fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55eb73056000 0x55eb7302fb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:8 7.38 7:1c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056800 con 0x55eb73056000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:9 7.4 7:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb73056000 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 3 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.851+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:10 7.24 7:24000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056c00 con 0x55eb72f38400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:11 7.14 7:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056800 con 0x55eb72f57400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:12 7.34 7:2c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb72f38400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 3 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.852+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:13 7.c 7:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057000 con 0x55eb72f57000 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 3 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:14 7.2c 7:34000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f56c00 con 0x55eb72f56800 2023-10-06T01:41:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 3 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:15 7.1c 7:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057400 con 0x55eb72f38400 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 4 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.853+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:16 7.3c 7:3c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57c00 con 0x55eb73056000 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 4 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:17 7.2 7:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057800 con 0x55eb72f57400 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 3 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9af5a7700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55eb73056800 0x55eb7302f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:18 7.22 7:44000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057800 con 0x55eb73056800 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.854+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55eb73056800 0x55eb7302f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9b45b1700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55eb73056800 0x55eb7302f600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.869 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:19 7.12 7:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056c00 con 0x55eb72f57000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 4 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:20 7.32 7:4c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056400 con 0x55eb73056000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 5 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.855+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:21 7.a 7:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057c00 con 0x55eb72f57400 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 4 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:22 7.2a 7:54000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb73056000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 6 ==== osd_op_reply(22 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:23 7.1a 7:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a000 con 0x55eb73056800 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(23 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.870 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:24 7.3a 7:5c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57800 con 0x55eb72f57400 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 5 ==== osd_op_reply(24 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.856+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:25 7.6 7:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a400 con 0x55eb73056000 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 7 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:26 7.26 7:64000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a000 con 0x55eb72f56800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 4 ==== osd_op_reply(26 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:27 7.16 7:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a800 con 0x55eb73056800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 3 ==== osd_op_reply(27 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:28 7.36 7:6c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306b800 con 0x55eb73056000 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 8 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.857+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:29 7.e 7:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306ac00 con 0x55eb72f56800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 5 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.871 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:30 7.2e 7:74000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306bc00 con 0x55eb73056800 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 4 ==== osd_op_reply(30 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9afda8700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55eb7306b000 0x55eb73030100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:31 7.1e 7:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306b400 con 0x55eb7306b000 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55eb7306b000 0x55eb73030100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.858+0000 7fe9b3db0700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55eb7306b000 0x55eb73030100 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== osd_op_reply(31 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb7306b000 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:32 7.3e 7:7c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57800 con 0x55eb72f38400 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 5 ==== osd_op_reply(32 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:33 7.1 7:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb73056800 2023-10-06T01:41:14.872 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 5 ==== osd_op_reply(33 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.873 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.859+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:34 7.21 7:84000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057800 con 0x55eb72f57400 2023-10-06T01:41:14.873 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.860+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 6 ==== osd_op_reply(34 [pgnls start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.873 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.860+0000 7fe9afda8700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55eb73057000 0x55eb73030680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.873 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.860+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:35 7.11 7:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057400 con 0x55eb73057000 2023-10-06T01:41:14.873 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.860+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55eb73057000 0x55eb73030680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.881 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.860+0000 7fe9b4db2700 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55eb73057000 0x55eb73030680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== osd_op_reply(35 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:36 7.31 7:8c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057c00 con 0x55eb72f57400 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 7 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:37 7.9 7:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056c00 con 0x55eb73056000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 9 ==== osd_op_reply(37 [pgnls start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:38 7.29 7:94000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a800 con 0x55eb72f57000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 5 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.861+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:39 7.19 7:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056400 con 0x55eb72f57000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 6 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.882 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:40 7.39 7:9c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306bc00 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:41 7.5 7:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f56c00 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 3 ==== osd_op_reply(41 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.862+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:42 7.25 7:a4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306b400 con 0x55eb73056800 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 6 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:43 7.15 7:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57c00 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 4 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:44 7.35 7:ac000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a400 con 0x55eb73056000 2023-10-06T01:41:14.883 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 10 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:45 7.d 7:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306b800 con 0x55eb72f57000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 7 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.863+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:46 7.2d 7:b4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057800 con 0x55eb72f57000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 8 ==== osd_op_reply(46 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:47 7.1d 7:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306ac00 con 0x55eb7306b000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== osd_op_reply(47 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb7306b000 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:48 7.3d 7:bc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057c00 con 0x55eb73056800 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 7 ==== osd_op_reply(48 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.864+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:49 7.3 7:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306bc00 con 0x55eb72f57400 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 8 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.884 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:50 7.23 7:c4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb72f57000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 9 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:51 7.13 7:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a000 con 0x55eb73056000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 11 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:52 7.33 7:cc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056400 con 0x55eb72f56800 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.865+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.2 v2:172.21.15.98:6824/120679 6 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb72f56800 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:53 7.b 7:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f56c00 con 0x55eb7306b000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.1 v2:172.21.15.98:6808/120656 3 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb7306b000 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:54 7.2b 7:d4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057800 con 0x55eb72f57400 2023-10-06T01:41:14.885 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.3 v2:172.21.15.98:6809/120655 9 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb72f57400 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:55 7.1b 7:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57c00 con 0x55eb73056000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 12 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.866+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:56 7.3b 7:dc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306a800 con 0x55eb72f38400 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.867+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.5 v2:172.21.15.184:6802/120874 6 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f38400 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.867+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:57 7.7 7:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73057400 con 0x55eb72f57000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.867+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 10 ==== osd_op_reply(57 [pgnls start_epoch 44 out=81b] v36'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.867+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:58 7.7 7:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38c00 con 0x55eb72f57000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.868+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 11 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.868+0000 7fe9bb343740 10 datascan.operator(): handling object 1099511627777.0 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.868+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:59 7.7 7:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f39000 con 0x55eb72f57000 2023-10-06T01:41:14.886 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.869+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 12 ==== osd_op_reply(59 10000000001.00000000 [call] v44'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.869+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:60 7.27 7:e4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f39400 con 0x55eb72f57000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 13 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:61 7.17 7:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73056c00 con 0x55eb73057000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 5 ==== osd_op_reply(61 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:62 7.37 7:ec000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb7306b800 con 0x55eb73056000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 13 ==== osd_op_reply(62 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.870+0000 7fe9afda8700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:63 7.f 7:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f38800 con 0x55eb73056000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.0 v2:172.21.15.98:6800/120654 14 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73056000 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:64 7.2f 7:f4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f57800 con 0x55eb73056800 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 8 ==== osd_op_reply(64 [pgnls start_epoch 44 out=81b] v36'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:65 7.2f 7:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f39800 con 0x55eb73056800 2023-10-06T01:41:14.887 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 <== osd.4 v2:172.21.15.184:6824/120887 9 ==== osd_op_reply(65 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eb72ea1440 con 0x55eb73056800 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9bb343740 10 datascan.operator(): handling object 1099511627777.1 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.871+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:66 7.7 7:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f39c00 con 0x55eb72f57000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.873+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 14 ==== osd_op_reply(66 10000000001.00000000 [call] v44'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.873+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:67 7.1f 7:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb72f39800 con 0x55eb72f57000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.873+0000 7fe9b3db0700 1 -- 172.21.15.98:0/1672934622 <== osd.6 v2:172.21.15.184:6800/120875 15 ==== osd_op_reply(67 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea0fc0 con 0x55eb72f57000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.873+0000 7fe9af5a7700 1 -- 172.21.15.98:0/1672934622 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:68 7.3f 7:fc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55eb73078000 con 0x55eb73057000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.873+0000 7fe9b4db2700 1 -- 172.21.15.98:0/1672934622 <== osd.7 v2:172.21.15.184:6816/120876 6 ==== osd_op_reply(68 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb72ea1200 con 0x55eb73057000 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55eb73056000 msgr2=0x55eb7302fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55eb73056000 0x55eb7302fb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55eb7306b000 msgr2=0x55eb73030100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.888 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55eb7306b000 0x55eb73030100 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55eb72f56800 msgr2=0x55eb7302e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55eb72f56800 0x55eb7302e580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55eb72f57400 msgr2=0x55eb7302eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55eb72f57400 0x55eb7302eb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 reap_dead start 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55eb73056800 msgr2=0x55eb7302f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55eb73056800 0x55eb7302f600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55eb72f38400 msgr2=0x55eb72e90100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55eb72f38400 0x55eb72e90100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55eb72f57000 msgr2=0x55eb7302f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55eb72f57000 0x55eb7302f080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.889 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55eb73057000 msgr2=0x55eb73030680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55eb73057000 0x55eb73030680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9b45b1700 1 -- 172.21.15.98:0/1672934622 reap_dead start 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 msgr2=0x55eb7302e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 0x55eb7302e000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55eb72e4c870 tx=0x55eb72e65bc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 msgr2=0x55eb72e90c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55eb72ef26c0 tx=0x55eb72edccc0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 shutdown_connections 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248400 0x55eb72e90c00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/1672934622 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55eb72f56400 0x55eb7302e000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 >> 172.21.15.98:0/1672934622 conn(0x55eb72248800 msgr2=0x55eb72e5b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.890 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 shutdown_connections 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/1672934622 wait complete. 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 msgr2=0x55eb72e8f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55eb72ea2630 tx=0x55eb72eca000 comp rx=0 tx=0).stop 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 shutdown_connections 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb72248c00 0x55eb72e8f600 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55eb72249800 0x55eb72e8f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 --2- 172.21.15.98:0/3506094302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb72249000 0x55eb72e8e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 >> 172.21.15.98:0/3506094302 conn(0x55eb72248000 msgr2=0x55eb72e5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 shutdown_connections 2023-10-06T01:41:14.891 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.874+0000 7fe9bb343740 1 -- 172.21.15.98:0/3506094302 wait complete. 2023-10-06T01:41:14.891 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', 'cephfs_data'] in time 0:00:00.128139, result: 2023-10-06T01:41:14.892 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --alternate-pool recovery_fs_metadata --filesystem cephfs --force-corrupt --force-init cephfs_data 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.960+0000 7f1d82a96740 1 -- start start 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.960+0000 7f1d82a96740 1 Processor -- start 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 0x55e9328c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 0x55e9328c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4fcc0 con 0x55e931c84400 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4f540 con 0x55e931c84800 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4fe00 con 0x55e931c84c00 2023-10-06T01:41:14.970 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35960/0 (socket says 172.21.15.98:35960) 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3887143792 learned_addr learned my addr 172.21.15.98:0/3887143792 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 0x55e9328c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 0x55e9328c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 msgr2=0x55e9328c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 0x55e9328c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 msgr2=0x55e9328c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 0x55e9328c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3887143792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e931b32340 con 0x55e931c84400 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 0x55e9328c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 0x55e9328c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.961+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e9328ec6c0 tx=0x55e9328a1440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6656a116697ac539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d794ff700 1 -- 172.21.15.98:0/3887143792 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b47600 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d794ff700 1 -- 172.21.15.98:0/3887143792 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e931b64f00 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d794ff700 1 -- 172.21.15.98:0/3887143792 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b471e0 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e931b32000 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e931b32ea0 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 4 waiting for MDS map... 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d794ff700 1 -- 172.21.15.98:0/3887143792 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55e931b52c40 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d794ff700 1 -- 172.21.15.98:0/3887143792 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x55e932904600 con 0x55e931c84400 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 4 Got MDS map 23 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 4 datascan.main: Using metadata pool output 2023-10-06T01:41:14.972 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 4 datascan.main: connecting to RADOS... 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 1 Processor -- start 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.962+0000 7f1d82a96740 1 -- start start 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85800 0x55e9328c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85c00 0x55e9328c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4fb80 con 0x55e93296c000 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4f540 con 0x55e931c85c00 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4edc0 con 0x55e931c85800 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7c505700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85800 0x55e9328c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.973 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35964/0 (socket says 172.21.15.98:35964) 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 -- 172.21.15.98:0/130834752 learned_addr learned my addr 172.21.15.98:0/130834752 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85c00 0x55e9328c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 -- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85800 msgr2=0x55e9328c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85800 0x55e9328c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 -- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85c00 msgr2=0x55e9328c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:14.974 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85c00 0x55e9328c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.963+0000 7f1d7b503700 1 -- 172.21.15.98:0/130834752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e931b33860 con 0x55e93296c000 2023-10-06T01:41:14.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d7b503700 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55e9328ed620 tx=0x55e931c86300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3beb9cdae3602911 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d774fb700 1 -- 172.21.15.98:0/130834752 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b47080 con 0x55e93296c000 2023-10-06T01:41:14.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d774fb700 1 -- 172.21.15.98:0/130834752 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e931b64000 con 0x55e93296c000 2023-10-06T01:41:14.975 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d774fb700 1 -- 172.21.15.98:0/130834752 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b47600 con 0x55e93296c000 2023-10-06T01:41:14.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 -- 172.21.15.98:0/130834752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 msgr2=0x55e9328c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55e9328ed620 tx=0x55e931c86300 comp rx=0 tx=0).stop 2023-10-06T01:41:14.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 -- 172.21.15.98:0/130834752 shutdown_connections 2023-10-06T01:41:14.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e93296c000 0x55e9328c8100 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.976 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85800 0x55e9328c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 --2- 172.21.15.98:0/130834752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85c00 0x55e9328c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 -- 172.21.15.98:0/130834752 >> 172.21.15.98:0/130834752 conn(0x55e931c85000 msgr2=0x55e932897800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 -- 172.21.15.98:0/130834752 shutdown_connections 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.964+0000 7f1d82a96740 1 -- 172.21.15.98:0/130834752 wait complete. 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 Processor -- start 2023-10-06T01:41:14.977 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 -- start start 2023-10-06T01:41:14.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85c00 0x55e9328c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85800 0x55e9328c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4e3c0 con 0x55e932976000 2023-10-06T01:41:14.978 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4e000 con 0x55e931c85800 2023-10-06T01:41:14.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d82a96740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e931b4f180 con 0x55e931c85c00 2023-10-06T01:41:14.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d7b503700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d7b503700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35976/0 (socket says 172.21.15.98:35976) 2023-10-06T01:41:14.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 learned_addr learned my addr 172.21.15.98:0/3066728252 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:14.979 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.965+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85c00 0x55e9328c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85800 0x55e9328c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85c00 msgr2=0x55e9328c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85c00 0x55e9328c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85800 msgr2=0x55e9328c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85800 0x55e9328c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e931b329c0 con 0x55e932976000 2023-10-06T01:41:14.980 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c85c00 0x55e9328c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c85800 0x55e9328c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55e9328ed890 tx=0x55e931c86240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d75cf8700 1 -- 172.21.15.98:0/3066728252 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b471e0 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d75cf8700 1 -- 172.21.15.98:0/3066728252 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e931b64300 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e931b33a00 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d75cf8700 1 -- 172.21.15.98:0/3066728252 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e931b46dc0 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e931b33860 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.966+0000 7f1d82a96740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.967+0000 7f1d75cf8700 1 -- 172.21.15.98:0/3066728252 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55e932885080 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d75cf8700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 0x55e9328c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d75cf8700 1 -- 172.21.15.98:0/3066728252 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x55e931b52a80 con 0x55e932976000 2023-10-06T01:41:14.981 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 0x55e9328c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d82a96740 4 datascan.main: data pool 'cephfs_data' has ID 7 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d82a96740 4 datascan.main: opening data pool 'cephfs_data' 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55e932976400 0x55e9328c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932976400 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 0x55e9328c9700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55e932889da0 tx=0x55e932913d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55e932976400 0x55e9328c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:14.982 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.968+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55e932976400 0x55e9328c7b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55e932976c00 0x55e9328c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932976c00 2023-10-06T01:41:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55e932976c00 0x55e9328c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55e932976c00 0x55e9328c7600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55e932977400 0x55e932a7e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.969+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:3 7.0 7:00000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932977400 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55e932977400 0x55e932a7e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55e932977400 0x55e932a7e000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d76cfa700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55e932978800 0x55e932a80680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:4 7.20 7:04000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932978c00 con 0x55e932978800 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.970+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55e932978800 0x55e932a80680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.004 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55e932978800 0x55e932a80680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d774fb700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55e932979400 0x55e932a80c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:5 7.10 7:08000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932979400 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55e932979400 0x55e932a80c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.971+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55e932979400 0x55e932a80c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.972+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.972+0000 7f1d76cfa700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55e932979000 0x55e932a81180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.972+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:6 7.30 7:0c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932979000 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.972+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55e932979000 0x55e932a81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.972+0000 7f1d7bd04700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55e932979000 0x55e932a81180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:7 7.8 7:10000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932978800 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:8 7.28 7:14000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932979000 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:9 7.18 7:18000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.973+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:10 7.38 7:1c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:11 7.4 7:20000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.006 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:12 7.24 7:24000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:13 7.14 7:28000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932979400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.974+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:14 7.34 7:2c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.975+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.975+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:15 7.c 7:30000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932979000 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.975+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.975+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:16 7.2c 7:34000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96c00 con 0x55e932978800 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:17 7.1c 7:38000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97000 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.007 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:18 7.3c 7:3c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96c00 con 0x55e932976c00 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:19 7.2 7:40000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932979400 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.976+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:20 7.22 7:44000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932976400 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:21 7.12 7:48000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932979000 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:22 7.32 7:4c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932976c00 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 6 ==== osd_op_reply(22 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.977+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:23 7.a 7:50000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96c00 con 0x55e932979400 2023-10-06T01:41:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:24 7.2a 7:54000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932978c00 con 0x55e932976c00 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 7 ==== osd_op_reply(24 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:25 7.1a 7:58000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932976400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:26 7.3a 7:5c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932979400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.978+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:27 7.6 7:60000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932976c00 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 8 ==== osd_op_reply(27 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:28 7.26 7:64000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932978800 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:29 7.16 7:68000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932976400 2023-10-06T01:41:15.009 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.979+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:30 7.36 7:6c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97400 con 0x55e932976c00 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:31 7.e 7:70000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932978800 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:32 7.2e 7:74000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97800 con 0x55e932976400 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d774fb700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55e932a96c00 0x55e932aa0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:33 7.1e 7:78000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97400 con 0x55e932a96c00 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.980+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55e932a96c00 0x55e932aa0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.981+0000 7f1d7b503700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55e932a96c00 0x55e932aa0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.981+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.981+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:34 7.3e 7:7c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97c00 con 0x55e932977400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.981+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.981+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:35 7.1 7:80000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97000 con 0x55e932976400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 6 ==== osd_op_reply(35 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:36 7.21 7:84000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932979400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d774fb700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 0x55e932aa0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:37 7.11 7:88000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932978c00 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 0x55e932aa0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.982+0000 7f1d7c505700 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 0x55e932aa0580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== osd_op_reply(37 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:38 7.31 7:8c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932979400 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:39 7.9 7:90000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932976c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 10 ==== osd_op_reply(39 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:40 7.29 7:94000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932979000 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 5 ==== osd_op_reply(40 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.983+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:41 7.19 7:98000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0000 con 0x55e932979000 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:42 7.39 7:9c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932978c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:43 7.5 7:a0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0400 con 0x55e932978c00 2023-10-06T01:41:15.012 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 3 ==== osd_op_reply(43 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:44 7.25 7:a4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0400 con 0x55e932976400 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 7 ==== osd_op_reply(44 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.984+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:45 7.15 7:a8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0800 con 0x55e932978c00 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:46 7.35 7:ac000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1c00 con 0x55e932976c00 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 11 ==== osd_op_reply(46 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:47 7.d 7:b0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0c00 con 0x55e932979000 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 7 ==== osd_op_reply(47 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.013 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.985+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:48 7.2d 7:b4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932979000 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 8 ==== osd_op_reply(48 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:49 7.1d 7:b8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1000 con 0x55e932a96c00 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== osd_op_reply(49 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:50 7.3d 7:bc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932976400 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 8 ==== osd_op_reply(50 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:51 7.3 7:c0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1400 con 0x55e932979400 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.986+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:52 7.23 7:c4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932979000 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:53 7.13 7:c8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1800 con 0x55e932976c00 2023-10-06T01:41:15.014 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 12 ==== osd_op_reply(53 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:54 7.33 7:cc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932978800 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 6 ==== osd_op_reply(54 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:55 7.b 7:d0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1000 con 0x55e932a96c00 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.987+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 3 ==== osd_op_reply(55 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:56 7.2b 7:d4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932979400 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:57 7.1b 7:d8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932976c00 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 13 ==== osd_op_reply(57 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:58 7.3b 7:dc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0000 con 0x55e932977400 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.015 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.988+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:59 7.7 7:e0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97400 con 0x55e932979000 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 44 out=81b] v44'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:60 7.0 7:00000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977c00 con 0x55e932977400 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:61 7.20 7:04000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97800 con 0x55e932978800 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 7 ==== osd_op_reply(61 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.989+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:62 7.10 7:08000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0400 con 0x55e932979400 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:63 7.30 7:0c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97c00 con 0x55e932979000 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:64 7.8 7:10000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932978800 2023-10-06T01:41:15.016 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 8 ==== osd_op_reply(64 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:65 7.28 7:14000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97000 con 0x55e932979000 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 12 ==== osd_op_reply(65 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.990+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:66 7.18 7:18000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0800 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 14 ==== osd_op_reply(66 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:67 7.38 7:1c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 15 ==== osd_op_reply(67 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:68 7.4 7:20000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1c00 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 16 ==== osd_op_reply(68 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:69 7.24 7:24000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932977400 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.017 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.991+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:70 7.14 7:28000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1400 con 0x55e932979400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 11 ==== osd_op_reply(70 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:71 7.34 7:2c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932977400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:72 7.c 7:30000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1800 con 0x55e932979000 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 13 ==== osd_op_reply(72 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.992+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:73 7.2c 7:34000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977c00 con 0x55e932978800 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 9 ==== osd_op_reply(73 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:74 7.1c 7:38000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0400 con 0x55e932977400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 10 ==== osd_op_reply(74 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:75 7.3c 7:3c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932976c00 2023-10-06T01:41:15.018 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 17 ==== osd_op_reply(75 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:76 7.2 7:40000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0c00 con 0x55e932979400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 12 ==== osd_op_reply(76 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.993+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:77 7.22 7:44000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97400 con 0x55e932976400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:78 7.12 7:48000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0000 con 0x55e932979000 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 14 ==== osd_op_reply(78 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:79 7.32 7:4c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932976c00 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 18 ==== osd_op_reply(79 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.994+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:80 7.a 7:50000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932979400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 13 ==== osd_op_reply(80 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:81 7.2a 7:54000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97c00 con 0x55e932976c00 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 19 ==== osd_op_reply(81 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:82 7.1a 7:58000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979800 con 0x55e932976400 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:83 7.3a 7:5c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97000 con 0x55e932979400 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.995+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 14 ==== osd_op_reply(83 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:84 7.6 7:60000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977000 con 0x55e932976c00 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 20 ==== osd_op_reply(84 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:85 7.26 7:64000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97400 con 0x55e932978800 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 10 ==== osd_op_reply(85 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:86 7.16 7:68000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932976400 2023-10-06T01:41:15.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 11 ==== osd_op_reply(86 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.996+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:87 7.36 7:6c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97800 con 0x55e932976c00 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 21 ==== osd_op_reply(87 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:88 7.e 7:70000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977800 con 0x55e932978800 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 11 ==== osd_op_reply(88 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:89 7.2e 7:74000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932976400 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.997+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:90 7.1e 7:78000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932977c00 con 0x55e932a96c00 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 4 ==== osd_op_reply(90 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:91 7.3e 7:7c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1800 con 0x55e932977400 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 11 ==== osd_op_reply(91 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.021 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:92 7.1 7:80000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932979c00 con 0x55e932976400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 13 ==== osd_op_reply(92 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.998+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:93 7.21 7:84000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0400 con 0x55e932979400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 15 ==== osd_op_reply(93 [pgnls-filter start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:94 7.11 7:88000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0800 con 0x55e932978c00 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 5 ==== osd_op_reply(94 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:95 7.31 7:8c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0000 con 0x55e932979400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 16 ==== osd_op_reply(95 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:14.999+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:96 7.9 7:90000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1c00 con 0x55e932976c00 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 22 ==== osd_op_reply(96 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:97 7.29 7:94000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1000 con 0x55e932979000 2023-10-06T01:41:15.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 15 ==== osd_op_reply(97 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:98 7.19 7:98000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1400 con 0x55e932979000 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 16 ==== osd_op_reply(98 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:99 7.39 7:9c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abc000 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.000+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:100 7.5 7:a0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab0c00 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 7 ==== osd_op_reply(100 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:101 7.25 7:a4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abc400 con 0x55e932976400 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 14 ==== osd_op_reply(101 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:102 7.15 7:a8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 8 ==== osd_op_reply(102 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.001+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:103 7.35 7:ac000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abc800 con 0x55e932976c00 2023-10-06T01:41:15.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.002+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 23 ==== osd_op_reply(103 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.002+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:104 7.d 7:b0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932979000 2023-10-06T01:41:15.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.002+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 17 ==== osd_op_reply(104 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.002+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:105 7.2d 7:b4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abcc00 con 0x55e932979000 2023-10-06T01:41:15.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.003+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 18 ==== osd_op_reply(105 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.003+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:106 7.1d 7:b8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1800 con 0x55e932a96c00 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.003+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 5 ==== osd_op_reply(106 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.003+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:107 7.3d 7:bc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abd000 con 0x55e932976400 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:108 7.3 7:c0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1000 con 0x55e932979400 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 17 ==== osd_op_reply(108 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:109 7.23 7:c4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abd400 con 0x55e932979000 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 19 ==== osd_op_reply(109 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:110 7.13 7:c8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1400 con 0x55e932976c00 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 24 ==== osd_op_reply(110 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.004+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- osd_op(unknown.0.0:111 7.33 7:cc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abd800 con 0x55e932978800 2023-10-06T01:41:15.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.2 v2:172.21.15.98:6824/120679 12 ==== osd_op_reply(111 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932978800 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:112 7.b 7:d0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96000 con 0x55e932a96c00 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 <== osd.1 v2:172.21.15.98:6808/120656 6 ==== osd_op_reply(112 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db440 con 0x55e932a96c00 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:113 7.2b 7:d4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abdc00 con 0x55e932979400 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.3 v2:172.21.15.98:6809/120655 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932979400 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:114 7.1b 7:d8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96800 con 0x55e932976c00 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 25 ==== osd_op_reply(114 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.005+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:115 7.3b 7:dc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abd800 con 0x55e932977400 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.5 v2:172.21.15.184:6802/120874 12 ==== osd_op_reply(115 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932977400 2023-10-06T01:41:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:116 7.7 7:e0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97c00 con 0x55e932979000 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 20 ==== osd_op_reply(116 [pgnls-filter start_epoch 44 out=81b] v44'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 10 datascan.operator(): handling object 10000000001.0 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:117 7.7 7:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6000 con 0x55e932979000 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 21 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (95) out=95b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+163 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.006+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:118 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6400 con 0x55e932978c00 2023-10-06T01:41:15.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d82a96740 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d82a96740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:119 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6800 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 10 ==== osd_op_reply(119 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d82a96740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.007+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:120 8.10 8:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6c00 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.008+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 11 ==== osd_op_reply(120 10000000000.00000000 [create,omap-set-header] v44'1 uv1 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.008+0000 7f1d82a96740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.008+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:121 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac7000 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.009+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 12 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.009+0000 7f1d82a96740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-06T01:41:15.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.009+0000 7f1d82a96740 20 datascan.read_dentry: sixmegs_head not found in result 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.009+0000 7f1d82a96740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.009+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:122 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=496b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac7400 con 0x55e932978c00 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 13 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v44'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:123 8.1f 8:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac7800 con 0x55e932976c00 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 26 ==== osd_op_reply(123 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-06T01:41:15.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:124 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac7c00 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 27 ==== osd_op_reply(124 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.010+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:125 8.1f 8:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6400 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 28 ==== osd_op_reply(125 1.00000000 [create,omap-set-header] v44'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:126 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6800 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 29 ==== osd_op_reply(126 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 20 datascan.read_dentry: subdir_head not found in result 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-10-06T01:41:15.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.013+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:127 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=495b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ac6c00 con 0x55e932976c00 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.015+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 30 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v44'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.015+0000 7f1d82a96740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.015+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:128 7.27 7:e4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a96400 con 0x55e932979000 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 22 ==== osd_op_reply(128 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:129 7.17 7:e8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97000 con 0x55e932978c00 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 14 ==== osd_op_reply(129 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:130 7.37 7:ec000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abc400 con 0x55e932976c00 2023-10-06T01:41:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 31 ==== osd_op_reply(130 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.016+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:131 7.f 7:f0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932a97800 con 0x55e932976c00 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.0 v2:172.21.15.98:6800/120654 32 ==== osd_op_reply(131 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932976c00 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:132 7.2f 7:f4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932abc000 con 0x55e932976400 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.4 v2:172.21.15.184:6824/120887 16 ==== osd_op_reply(132 [pgnls-filter start_epoch 44 out=49b] v36'9 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932976400 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d76cfa700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:133 7.1f 7:f8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932976800 con 0x55e932979000 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d7bd04700 1 -- 172.21.15.98:0/3066728252 <== osd.6 v2:172.21.15.184:6800/120875 23 ==== osd_op_reply(133 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328dafc0 con 0x55e932979000 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.017+0000 7f1d774fb700 1 -- 172.21.15.98:0/3066728252 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:134 7.3f 7:fc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e932ab1800 con 0x55e932978c00 2023-10-06T01:41:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d7c505700 1 -- 172.21.15.98:0/3066728252 <== osd.7 v2:172.21.15.184:6816/120876 15 ==== osd_op_reply(134 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e9328db200 con 0x55e932978c00 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55e932976c00 msgr2=0x55e9328c7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x55e932976c00 0x55e9328c7600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55e932a96c00 msgr2=0x55e932aa0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x55e932a96c00 0x55e932aa0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55e932978800 msgr2=0x55e932a80680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x55e932978800 0x55e932a80680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55e932979400 msgr2=0x55e932a80c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x55e932979400 0x55e932a80c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55e932976400 msgr2=0x55e9328c7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x55e932976400 0x55e9328c7b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55e932977400 msgr2=0x55e932a7e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x55e932977400 0x55e932a7e000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55e932979000 msgr2=0x55e932a81180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x55e932979000 0x55e932a81180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 reap_dead start 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 msgr2=0x55e932aa0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 0x55e932aa0580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d7b503700 1 -- 172.21.15.98:0/3066728252 reap_dead start 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 msgr2=0x55e9328c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 0x55e9328c9700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55e932889da0 tx=0x55e932913d40 comp rx=0 tx=0).stop 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 msgr2=0x55e9328c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55e9328ed890 tx=0x55e931c86240 comp rx=0 tx=0).stop 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 shutdown_connections 2023-10-06T01:41:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x55e932978c00 0x55e932aa0580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e932976000 0x55e9328c8100 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3066728252 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x55e932978400 0x55e9328c9700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 >> 172.21.15.98:0/3066728252 conn(0x55e931c85000 msgr2=0x55e932897800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 shutdown_connections 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3066728252 wait complete. 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 msgr2=0x55e9328c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e9328ec6c0 tx=0x55e9328a1440 comp rx=0 tx=0).stop 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 shutdown_connections 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e931c84c00 0x55e9328c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e931c84400 0x55e9328c6580 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 --2- 172.21.15.98:0/3887143792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e931c84800 0x55e9328c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.018+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 >> 172.21.15.98:0/3887143792 conn(0x55e931c84000 msgr2=0x55e932897000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.019+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 shutdown_connections 2023-10-06T01:41:15.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.019+0000 7f1d82a96740 1 -- 172.21.15.98:0/3887143792 wait complete. 2023-10-06T01:41:15.046 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', '--force-corrupt', '--force-init', 'cephfs_data'] in time 0:00:00.154321, result: 2023-10-06T01:41:15.046 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links --filesystem recovery_fs 2023-10-06T01:41:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.106+0000 7f49d69c4740 1 -- start start 2023-10-06T01:41:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.106+0000 7f49d69c4740 1 Processor -- start 2023-10-06T01:41:15.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.106+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec2400 0x5645e0afa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 0x5645e0afab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8dcc0 con 0x5645dfec2c00 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8d540 con 0x5645dfec2400 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8de00 con 0x5645dfec2800 2023-10-06T01:41:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35986/0 (socket says 172.21.15.98:35986) 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 -- 172.21.15.98:0/1698112317 learned_addr learned my addr 172.21.15.98:0/1698112317 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cf431700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 0x5645e0afab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cfc32700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec2400 0x5645e0afa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 -- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 msgr2=0x5645e0afab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 0x5645e0afab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 -- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec2400 msgr2=0x5645e0afa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec2400 0x5645e0afa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 -- 172.21.15.98:0/1698112317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645dfd70b60 con 0x5645dfec2c00 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cf431700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 0x5645e0afab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d0433700 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5645e0b26690 tx=0x5645dfec4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4626cf362341d62a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cd42d700 1 -- 172.21.15.98:0/1698112317 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd85080 con 0x5645dfec2c00 2023-10-06T01:41:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cd42d700 1 -- 172.21.15.98:0/1698112317 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645dfda2f00 con 0x5645dfec2c00 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cd42d700 1 -- 172.21.15.98:0/1698112317 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd85600 con 0x5645dfec2c00 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645dfd70000 con 0x5645dfec2c00 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5645dfd70ea0 con 0x5645dfec2c00 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49d69c4740 4 waiting for MDS map... 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cd42d700 1 -- 172.21.15.98:0/1698112317 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x5645dfd90c40 con 0x5645dfec2c00 2023-10-06T01:41:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.107+0000 7f49cd42d700 1 -- 172.21.15.98:0/1698112317 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x5645e0b3b200 con 0x5645dfec2c00 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.108+0000 7f49d69c4740 4 Got MDS map 23 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.108+0000 7f49d69c4740 4 datascan.main: Using metadata pool output 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.108+0000 7f49d69c4740 4 datascan.main: connecting to RADOS... 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.108+0000 7f49d69c4740 1 Processor -- start 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.108+0000 7f49d69c4740 1 -- start start 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8db80 con 0x5645e0b98000 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8d540 con 0x5645dfec3c00 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8dcc0 con 0x5645dfec3800 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35988/0 (socket says 172.21.15.98:35988) 2023-10-06T01:41:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 -- 172.21.15.98:0/4029769105 learned_addr learned my addr 172.21.15.98:0/4029769105 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cfc32700 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 -- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 msgr2=0x5645e0afb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 -- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 msgr2=0x5645e0afbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 -- 172.21.15.98:0/4029769105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645dfd709c0 con 0x5645e0b98000 2023-10-06T01:41:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cf431700 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5645e0b275c0 tx=0x5645e0b4acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6171524da7e11f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cb429700 1 -- 172.21.15.98:0/4029769105 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd85080 con 0x5645e0b98000 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cb429700 1 -- 172.21.15.98:0/4029769105 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645dfda2600 con 0x5645e0b98000 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.109+0000 7f49cb429700 1 -- 172.21.15.98:0/4029769105 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd85600 con 0x5645e0b98000 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- 172.21.15.98:0/4029769105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 msgr2=0x5645e0afc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5645e0b275c0 tx=0x5645e0b4acc0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- 172.21.15.98:0/4029769105 shutdown_connections 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0b98000 0x5645e0afc100 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 --2- 172.21.15.98:0/4029769105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- 172.21.15.98:0/4029769105 >> 172.21.15.98:0/4029769105 conn(0x5645dfec3000 msgr2=0x5645e0ad5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- 172.21.15.98:0/4029769105 shutdown_connections 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- 172.21.15.98:0/4029769105 wait complete. 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 Processor -- start 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.110+0000 7f49d69c4740 1 -- start start 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 0x5645e0afc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8cdc0 con 0x5645e0bba000 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8c3c0 con 0x5645dfec3c00 2023-10-06T01:41:15.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49d69c4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645dfd8c000 con 0x5645dfec3800 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 0x5645e0afc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 0x5645e0afc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35990/0 (socket says 172.21.15.98:35990) 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 learned_addr learned my addr 172.21.15.98:0/2614822856 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 msgr2=0x5645e0afc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec3800 0x5645e0afc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 msgr2=0x5645e0afcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec3c00 0x5645e0afcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645dfd71860 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 0x5645e0afc100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5645e0b27860 tx=0x5645e0ad8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.111+0000 7f49c9c26700 1 -- 172.21.15.98:0/2614822856 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd84dc0 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.112+0000 7f49c9c26700 1 -- 172.21.15.98:0/2614822856 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645dfda2000 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.112+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645dfd70b60 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.112+0000 7f49c9c26700 1 -- 172.21.15.98:0/2614822856 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645dfd85080 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.112+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645dfd70000 con 0x5645e0bba000 2023-10-06T01:41:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.112+0000 7f49d69c4740 4 datascan.init: resolving metadata pool 8 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49c9c26700 1 -- 172.21.15.98:0/2614822856 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x5645e0ac3080 con 0x5645e0bba000 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49c9c26700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x5645e0bcc400 0x5645e0afd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49c9c26700 1 -- 172.21.15.98:0/2614822856 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x5645dfd90a80 con 0x5645e0bba000 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49d69c4740 4 datascan.init: found metadata pool 'recovery_fs_metadata' 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49d69c4740 4 datascan.main: resolving metadata pool 8 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x5645e0bcc400 0x5645e0afd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x5645e0bba400 0x5645e0afbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:1 8.0 8:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bba800 con 0x5645e0bba400 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x5645e0bba400 0x5645e0afbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.113+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x5645e0bcc400 0x5645e0afd700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5645e0ac7cb0 tx=0x5645e0b4be60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x5645e0bba400 0x5645e0afbb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.136 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x5645e0bba800 0x5645e0cbc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:2 8.10 8:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcc800 con 0x5645e0bba800 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49d0433700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x5645e0bba800 0x5645e0cbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.114+0000 7f49d0433700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x5645e0bba800 0x5645e0cbc000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.115+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=81b] v44'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.115+0000 7f49d0433700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x5645e0bcc800 0x5645e0cbc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.115+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:3 8.8 8:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcd000 con 0x5645e0bcc800 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.115+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x5645e0bcc800 0x5645e0cbc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.115+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x5645e0bcc800 0x5645e0cbc580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.137 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x5645e0bcd000 0x5645e0cbcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:4 8.18 8:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcd800 con 0x5645e0bcd000 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x5645e0bcd000 0x5645e0cbcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x5645e0bcd000 0x5645e0cbcb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.116+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:5 8.4 8:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcd800 con 0x5645e0bcc800 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:6 8.14 8:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcd800 con 0x5645e0bcd000 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(6 [pgnls start_epoch 44 out=77b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x5645e0bcd800 0x5645e0cbd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.138 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:7 8.c 8:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bccc00 con 0x5645e0bcd800 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49d0433700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x5645e0bcd800 0x5645e0cbd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.117+0000 7f49d0433700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x5645e0bcd800 0x5645e0cbd080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:8 8.1c 8:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bccc00 con 0x5645e0bcc800 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 3 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:9 8.2 8:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bccc00 con 0x5645e0bcd000 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 3 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x5645e0bccc00 0x5645e0cbd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:10 8.12 8:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bcdc00 con 0x5645e0bccc00 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.118+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x5645e0bccc00 0x5645e0cbd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.139 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x5645e0bccc00 0x5645e0cbd600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 1 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cfc32700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x5645e0bcdc00 0x5645e0cbdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:11 8.a 8:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcdc00 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x5645e0bcdc00 0x5645e0cbdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.119+0000 7f49cf431700 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x5645e0bcdc00 0x5645e0cbdb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:12 8.1a 8:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd800 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.140 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:13 8.6 8:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd800 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 3 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:14 8.16 8:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.120+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:15 8.e 8:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd800 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 4 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:16 8.1e 8:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.141 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 3 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:17 8.1 8:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcdc00 2023-10-06T01:41:15.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:18 8.11 8:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcc800 2023-10-06T01:41:15.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 4 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.142 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.121+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:19 8.9 8:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bccc00 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 2 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:20 8.19 8:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcc800 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 5 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:21 8.5 8:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bccc00 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 3 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.122+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:22 8.15 8:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba800 2023-10-06T01:41:15.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== osd_op_reply(22 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:23 8.d 8:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcc800 2023-10-06T01:41:15.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 6 ==== osd_op_reply(23 [pgnls start_epoch 44 out=74b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:24 8.1d 8:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd800 2023-10-06T01:41:15.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 5 ==== osd_op_reply(24 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.144 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:25 8.3 8:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcdc00 2023-10-06T01:41:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 3 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.123+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:26 8.13 8:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 4 ==== osd_op_reply(26 [pgnls start_epoch 44 out=98b] v44'2 uv2 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:27 8.b 8:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 5 ==== osd_op_reply(27 [pgnls start_epoch 44 out=76b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:28 8.1b 8:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd000 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 4 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.124+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:29 8.7 8:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcdc00 2023-10-06T01:41:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 4 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:30 8.17 8:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba800 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 3 ==== osd_op_reply(30 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:31 8.f 8:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 6 ==== osd_op_reply(31 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.125+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:32 8.1f 8:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bcd800 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 6 ==== osd_op_reply(32 [pgnls start_epoch 44 out=101b] v44'3 uv3 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbac00 con 0x5645e0bba800 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 4 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=497b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+497 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 20 datascan.scan_links: injected ino detected 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-06T01:41:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:34 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb000 con 0x5645e0bba400 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 7 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.126+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:35 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb400 con 0x5645e0bcd800 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=496b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+496 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49d69c4740 20 datascan.scan_links: injected ino detected 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:36 8.0 8:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 8 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:37 8.10 8:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba800 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 5 ==== osd_op_reply(37 [pgnls start_epoch 44 out=81b] v44'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:38 8.8 8:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.127+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 7 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:39 8.18 8:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd000 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 5 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:40 8.4 8:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 8 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:41 8.14 8:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd000 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 6 ==== osd_op_reply(41 [pgnls start_epoch 44 out=77b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.128+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:42 8.c 8:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 8 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:43 8.1c 8:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 9 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:44 8.2 8:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd000 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 7 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:45 8.12 8:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bccc00 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 4 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.129+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:46 8.a 8:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcdc00 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 5 ==== osd_op_reply(46 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:47 8.1a 8:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 9 ==== osd_op_reply(47 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:48 8.6 8:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 10 ==== osd_op_reply(48 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:49 8.16 8:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 9 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.130+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:50 8.e 8:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 11 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:51 8.1e 8:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 10 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:52 8.1 8:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcdc00 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 6 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:53 8.11 8:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 10 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.131+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:54 8.9 8:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bccc00 2023-10-06T01:41:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 5 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:55 8.19 8:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 11 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- osd_op(unknown.0.0:56 8.5 8:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bccc00 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.6 v2:172.21.15.184:6800/120875 6 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bccc00 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:57 8.15 8:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 6 ==== osd_op_reply(57 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.132+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:58 8.d 8:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcc800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 12 ==== osd_op_reply(58 [pgnls start_epoch 44 out=74b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:59 8.1d 8:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 12 ==== osd_op_reply(59 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:60 8.3 8:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcdc00 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 7 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:61 8.13 8:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 11 ==== osd_op_reply(61 [pgnls start_epoch 44 out=98b] v44'2 uv2 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.133+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:62 8.b 8:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 12 ==== osd_op_reply(62 [pgnls start_epoch 44 out=76b] v44'1 uv1 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:63 8.1b 8:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd000 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.4 v2:172.21.15.184:6824/120887 8 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcd000 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:64 8.7 8:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcdc00 2023-10-06T01:41:15.168 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.1 v2:172.21.15.98:6808/120656 8 ==== osd_op_reply(64 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bcdc00 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.134+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:65 8.17 8:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 7 ==== osd_op_reply(65 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:66 8.f 8:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bba400 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 13 ==== osd_op_reply(66 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:67 8.1f 8:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb800 con 0x5645e0bcd800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 13 ==== osd_op_reply(67 [pgnls start_epoch 44 out=101b] v44'3 uv3 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:68 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbbc00 con 0x5645e0bba800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 8 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=497b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+497 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 20 datascan.scan_links: injected ino detected 2023-10-06T01:41:15.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.135+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:69 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0000 con 0x5645e0bba400 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 14 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-06T01:41:15.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:70 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0800 con 0x5645e0bcd800 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=496b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+496 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 20 datascan.scan_links: injected ino detected 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: processing 2 injected_inos 2023-10-06T01:41:15.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:71 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd0c00 con 0x5645e0bcd800 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=495b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+495 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 20 datascan.scan_links: first is now 2 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.136+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:72 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=495b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd1000 con 0x5645e0bcd800 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.0 v2:172.21.15.98:6800/120654 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v44'4 uv4 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bcd800 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-06T01:41:15.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:73 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd1400 con 0x5645e0bba800 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 9 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=496b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+496 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 20 datascan.scan_links: first is now 2 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.140+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- osd_op(unknown.0.0:74 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=496b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd1800 con 0x5645e0bba800 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d0433700 1 -- 172.21.15.98:0/2614822856 <== osd.7 v2:172.21.15.184:6816/120876 10 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v44'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5645e0b25200 con 0x5645e0bba800 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 10 datascan.scan_links: updating inotable 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:75 8.13 8:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0cd1c00 con 0x5645e0bba400 2023-10-06T01:41:15.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 15 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.142+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:76 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbac00 con 0x5645e0bba400 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.143+0000 7f49cf431700 1 -- 172.21.15.98:0/2614822856 <== osd.3 v2:172.21.15.98:6809/120655 16 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v44'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5645e0b24fc0 con 0x5645e0bba400 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.143+0000 7f49d69c4740 10 datascan.scan_links: updating snaptable 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.143+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- osd_op(unknown.0.0:77 8.d 8:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5645e0bbb400 con 0x5645e0bcc800 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 <== osd.5 v2:172.21.15.184:6802/120874 13 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5645e0b25440 con 0x5645e0bcc800 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x5645e0bcd800 msgr2=0x5645e0cbd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x5645e0bcd800 0x5645e0cbd080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x5645e0bcdc00 msgr2=0x5645e0cbdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x5645e0bcdc00 0x5645e0cbdb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x5645e0bba400 msgr2=0x5645e0afbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x5645e0bba400 0x5645e0afbb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x5645e0bcd000 msgr2=0x5645e0cbcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x5645e0bcd000 0x5645e0cbcb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 reap_dead start 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x5645e0bcc800 msgr2=0x5645e0cbc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x5645e0bcc800 0x5645e0cbc580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 reap_dead start 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x5645e0bccc00 msgr2=0x5645e0cbd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x5645e0bccc00 0x5645e0cbd600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x5645e0bba800 msgr2=0x5645e0cbc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x5645e0bba800 0x5645e0cbc000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x5645e0bcc400 msgr2=0x5645e0afd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x5645e0bcc400 0x5645e0afd700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5645e0ac7cb0 tx=0x5645e0b4be60 comp rx=0 tx=0).stop 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 msgr2=0x5645e0afc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 --2- 172.21.15.98:0/2614822856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645e0bba000 0x5645e0afc100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5645e0b27860 tx=0x5645e0ad8a80 comp rx=0 tx=0).stop 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49cfc32700 1 -- 172.21.15.98:0/2614822856 reap_dead start 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 shutdown_connections 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 >> 172.21.15.98:0/2614822856 conn(0x5645dfec3000 msgr2=0x5645e0ad5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 shutdown_connections 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/2614822856 wait complete. 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.144+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 msgr2=0x5645e0afb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5645e0b26690 tx=0x5645dfec4300 comp rx=0 tx=0).stop 2023-10-06T01:41:15.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 shutdown_connections 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5645dfec2c00 0x5645e0afb080 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5645dfec2400 0x5645e0afa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 --2- 172.21.15.98:0/1698112317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5645dfec2800 0x5645e0afab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 >> 172.21.15.98:0/1698112317 conn(0x5645dfec2000 msgr2=0x5645e0ad5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 shutdown_connections 2023-10-06T01:41:15.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.145+0000 7f49d69c4740 1 -- 172.21.15.98:0/1698112317 wait complete. 2023-10-06T01:41:15.177 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links', '--filesystem', 'recovery_fs'] in time 0:00:00.130870, result: 2023-10-06T01:41:15.177 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 event recover_dentries list --alternate-pool recovery_fs_metadata 2023-10-06T01:41:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 -- start start 2023-10-06T01:41:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 Processor -- start 2023-10-06T01:41:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 0x5625470c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625470dec00 0x5625470c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635bcc0 con 0x5625470de800 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635b540 con 0x5625470dec00 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254635bb80 con 0x5625470de400 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.235+0000 7fc9999b6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36012/0 (socket says 172.21.15.98:36012) 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 -- 172.21.15.98:0/451870000 learned_addr learned my addr 172.21.15.98:0/451870000 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625470dec00 0x5625470c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 0x5625470c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 -- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 msgr2=0x5625470c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 0x5625470c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 -- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625470dec00 msgr2=0x5625470c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625470dec00 0x5625470c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 -- 172.21.15.98:0/451870000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254633e340 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 0x5625470c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9999b6700 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5625470980c0 tx=0x562547104000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9073e93916027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9979b2700 1 -- 172.21.15.98:0/451870000 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562546353600 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9979b2700 1 -- 172.21.15.98:0/451870000 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562546370480 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9979b2700 1 -- 172.21.15.98:0/451870000 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625463531e0 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56254633e000 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56254633eea0 con 0x5625470de800 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 4 waiting for MDS map... 2023-10-06T01:41:15.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9979b2700 1 -- 172.21.15.98:0/451870000 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x56254635ec40 con 0x5625470de800 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9979b2700 1 -- 172.21.15.98:0/451870000 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x562547102600 con 0x5625470de800 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 4 Got MDS map 23 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 10 main: JournalTool::main 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.236+0000 7fc9a0ce9240 4 main: JournalTool: connecting to RADOS... 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.237+0000 7fc9a0ce9240 1 Processor -- start 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.237+0000 7fc9a0ce9240 1 -- start start 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254716e000 0x5625470ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635adc0 con 0x5625470df800 2023-10-06T01:41:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635b540 con 0x56254716e000 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254635a3c0 con 0x5625470dfc00 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36018/0 (socket says 172.21.15.98:36018) 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 -- 172.21.15.98:0/825098528 learned_addr learned my addr 172.21.15.98:0/825098528 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9999b6700 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254716e000 0x5625470ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 -- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 msgr2=0x5625470c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 -- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254716e000 msgr2=0x5625470ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254716e000 0x5625470ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 -- 172.21.15.98:0/825098528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254633f860 con 0x5625470df800 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5625470995c0 tx=0x5625470adb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f3f83032879926 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9959ae700 1 -- 172.21.15.98:0/825098528 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562546353080 con 0x5625470df800 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9959ae700 1 -- 172.21.15.98:0/825098528 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562546370c00 con 0x5625470df800 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9959ae700 1 -- 172.21.15.98:0/825098528 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562546353600 con 0x5625470df800 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/825098528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 msgr2=0x5625470c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5625470995c0 tx=0x5625470adb60 comp rx=0 tx=0).stop 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.238+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/825098528 shutdown_connections 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254716e000 0x5625470ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470c9600 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/825098528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/825098528 >> 172.21.15.98:0/825098528 conn(0x5625470df000 msgr2=0x5625470a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/825098528 shutdown_connections 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/825098528 wait complete. 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 Processor -- start 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.239+0000 7fc9a0ce9240 1 -- start start 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254717e000 0x5625470ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635a000 con 0x5625470df800 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254635b180 con 0x56254717e000 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254635b040 con 0x5625470dfc00 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36028/0 (socket says 172.21.15.98:36028) 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 -- 172.21.15.98:0/2870073947 learned_addr learned my addr 172.21.15.98:0/2870073947 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9999b6700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254717e000 0x5625470ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 msgr2=0x5625470cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470dfc00 0x5625470cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254717e000 msgr2=0x5625470ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56254717e000 0x5625470ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 -- 172.21.15.98:0/2870073947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254633e9c0 con 0x5625470df800 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc99a1b7700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x5625471800c0 tx=0x562547110c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9941ab700 1 -- 172.21.15.98:0/2870073947 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562546353080 con 0x5625470df800 2023-10-06T01:41:15.249 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9941ab700 1 -- 172.21.15.98:0/2870073947 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562546370f00 con 0x5625470df800 2023-10-06T01:41:15.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56254633fa00 con 0x5625470df800 2023-10-06T01:41:15.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9941ab700 1 -- 172.21.15.98:0/2870073947 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562546353600 con 0x5625470df800 2023-10-06T01:41:15.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56254633e9c0 con 0x5625470df800 2023-10-06T01:41:15.250 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.240+0000 7fc9a0ce9240 4 main: JournalTool: resolving pool 8 2023-10-06T01:41:15.253 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.241+0000 7fc9941ab700 1 -- 172.21.15.98:0/2870073947 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x562547091080 con 0x5625470df800 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9941ab700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x562547186400 0x5625470cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9941ab700 1 -- 172.21.15.98:0/2870073947 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x56254635ea80 con 0x5625470df800 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9a0ce9240 4 main: JournalTool: creating IoCtx.. 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9a0ce9240 4 main: Executing for rank 0 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x562547186400 0x5625470cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9a0ce9240 1 main_event: Using alternate pool recovery_fs_metadata 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x56254717e400 0x5625470c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:1 8.f 8:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x56254717e800 con 0x56254717e400 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9999b6700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x56254717e400 0x5625470c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.254 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x562547186400 0x5625470cb700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5625470995c0 tx=0x5625470adb00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9999b6700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x56254717e400 0x5625470c9b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.242+0000 7fc9999b6700 1 -- 172.21.15.98:0/2870073947 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(1 400.00000000 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5625470f2fc0 con 0x56254717e400 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc9a0ce9240 -1 Pointer 400.00000000 is absent 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc9a0ce9240 4 main_event: consumed 0 inodes 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x56254717ec00 0x5625470c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- osd_op(unknown.0.0:2 8.3 8:c0ce80d4:::4.00000000:head [omap-rm-keys in=4b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x56254717f000 con 0x56254717ec00 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x56254717ec00 0x5625470c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.243+0000 7fc99a9b8700 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x56254717ec00 0x5625470c9600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.255 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc99a9b8700 1 -- 172.21.15.98:0/2870073947 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== osd_op_reply(2 4.00000000 [omap-rm-keys] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5625470f3680 con 0x56254717ec00 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 4 0 events 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x56254717ec00 msgr2=0x5625470c9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x56254717ec00 0x5625470c9600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x56254717e400 msgr2=0x5625470c9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x56254717e400 0x5625470c9b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x562547186400 msgr2=0x5625470cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x562547186400 0x5625470cb700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5625470995c0 tx=0x5625470adb00 comp rx=0 tx=0).stop 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 msgr2=0x5625470ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x5625471800c0 tx=0x562547110c00 comp rx=0 tx=0).stop 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc99a1b7700 1 -- 172.21.15.98:0/2870073947 reap_dead start 2023-10-06T01:41:15.256 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 shutdown_connections 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/2870073947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470df800 0x5625470ca680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 >> 172.21.15.98:0/2870073947 conn(0x5625470df000 msgr2=0x5625470a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 shutdown_connections 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/2870073947 wait complete. 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 msgr2=0x5625470c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5625470980c0 tx=0x562547104000 comp rx=0 tx=0).stop 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 shutdown_connections 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625470dec00 0x5625470c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5625470de400 0x5625470c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 --2- 172.21.15.98:0/451870000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625470de800 0x5625470c8b00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 >> 172.21.15.98:0/451870000 conn(0x5625470de000 msgr2=0x5625470a3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 shutdown_connections 2023-10-06T01:41:15.257 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.244+0000 7fc9a0ce9240 1 -- 172.21.15.98:0/451870000 wait complete. 2023-10-06T01:41:15.258 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'event', 'recover_dentries', 'list', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.080282, result: 2023-10-06T01:41:15.258 DEBUG:teuthology.orchestra.run.smithi098:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 journal reset --force 2023-10-06T01:41:15.329 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.328+0000 7ff3a4f6f240 1 -- start start 2023-10-06T01:41:15.329 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.328+0000 7ff3a4f6f240 1 Processor -- start 2023-10-06T01:41:15.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 0x559e2da1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3cc0 con 0x559e2da22c00 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3540 con 0x559e2da22800 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3e00 con 0x559e2da22400 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39e43d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39e43d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:39932/0 (socket says 172.21.15.98:39932) 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39e43d700 1 -- 172.21.15.98:0/567949634 learned_addr learned my addr 172.21.15.98:0/567949634 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36038/0 (socket says 172.21.15.98:36038) 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 0x559e2da1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 -- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 msgr2=0x559e2da1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 -- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 msgr2=0x559e2da1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 0x559e2da1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 -- 172.21.15.98:0/567949634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2cc86340 con 0x559e2da22c00 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39e43d700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 0x559e2da1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:15.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559e2d9e0060 tx=0x559e2da521e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3537c911ee050f17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39bc38700 1 -- 172.21.15.98:0/567949634 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b600 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39bc38700 1 -- 172.21.15.98:0/567949634 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e2ccb8900 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39bc38700 1 -- 172.21.15.98:0/567949634 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b1e0 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e2cc86000 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559e2cc86ea0 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff3a4f6f240 4 waiting for MDS map... 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.329+0000 7ff39bc38700 1 -- 172.21.15.98:0/567949634 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x559e2cca6c40 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff39bc38700 1 -- 172.21.15.98:0/567949634 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x559e2da5e600 con 0x559e2da22c00 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff3a4f6f240 4 Got MDS map 23 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff3a4f6f240 10 main: JournalTool::main 2023-10-06T01:41:15.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff3a4f6f240 4 main: JournalTool: connecting to RADOS... 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff3a4f6f240 1 Processor -- start 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.330+0000 7ff3a4f6f240 1 -- start start 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0000 0x559e2da1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3b80 con 0x559e2dab0800 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3540 con 0x559e2dab0000 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca2dc0 con 0x559e2dab0400 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36048/0 (socket says 172.21.15.98:36048) 2023-10-06T01:41:15.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4079969702 learned_addr learned my addr 172.21.15.98:0/4079969702 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39e43d700 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0000 0x559e2da1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 msgr2=0x559e2da1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0000 msgr2=0x559e2da1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0000 0x559e2da1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4079969702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2cc87860 con 0x559e2dab0800 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x559e2d9e1560 tx=0x559e2da64cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409c33389b68010 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff399c34700 1 -- 172.21.15.98:0/4079969702 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b600 con 0x559e2dab0800 2023-10-06T01:41:15.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff399c34700 1 -- 172.21.15.98:0/4079969702 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e2ccb8300 con 0x559e2dab0800 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.331+0000 7ff399c34700 1 -- 172.21.15.98:0/4079969702 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b1e0 con 0x559e2dab0800 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4079969702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 msgr2=0x559e2da1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x559e2d9e1560 tx=0x559e2da64cc0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4079969702 shutdown_connections 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0000 0x559e2da1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4079969702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4079969702 >> 172.21.15.98:0/4079969702 conn(0x559e2da23000 msgr2=0x559e2d9eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4079969702 shutdown_connections 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4079969702 wait complete. 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 Processor -- start 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.332+0000 7ff3a4f6f240 1 -- start start 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca23c0 con 0x559e2dab0000 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca2000 con 0x559e2dab0800 2023-10-06T01:41:15.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3180 con 0x559e2dab0400 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36052/0 (socket says 172.21.15.98:36052) 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 -- 172.21.15.98:0/436502718 learned_addr learned my addr 172.21.15.98:0/436502718 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39e43d700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 -- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 msgr2=0x559e2da1c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 -- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0800 msgr2=0x559e2da1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 -- 172.21.15.98:0/436502718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2cc86340 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x559e2dada060 tx=0x559e2da520c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.333+0000 7ff398431700 1 -- 172.21.15.98:0/436502718 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9adc0 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.334+0000 7ff398431700 1 -- 172.21.15.98:0/436502718 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e2ccb8c00 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.334+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559e2cc87a00 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.334+0000 7ff398431700 1 -- 172.21.15.98:0/436502718 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b600 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.334+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e2cc87860 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.334+0000 7ff3a4f6f240 4 main: JournalTool: resolving pool 8 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff398431700 1 -- 172.21.15.98:0/436502718 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x559e2d9d9080 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff398431700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 0x559e2da1d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff398431700 1 -- 172.21.15.98:0/436502718 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x559e2cca6a80 con 0x559e2dab0000 2023-10-06T01:41:15.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff39e43d700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 0x559e2da1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff3a4f6f240 4 main: JournalTool: creating IoCtx.. 2023-10-06T01:41:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff3a4f6f240 4 main: Executing for rank 0 2023-10-06T01:41:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.335+0000 7ff39e43d700 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 0x559e2da1d700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x559e2ccf1e00 tx=0x559e2d9f5980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 -- start start 2023-10-06T01:41:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 Processor -- start 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab1000 0x559e2da1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dbb6400 0x559e2dc16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca3180 con 0x559e2dab1400 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca2000 con 0x559e2dbb6400 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.336+0000 7ff3a4f6f240 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e2cca2dc0 con 0x559e2dab1000 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36060/0 (socket says 172.21.15.98:36060) 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 -- 172.21.15.98:0/4003104471 learned_addr learned my addr 172.21.15.98:0/4003104471 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab1000 0x559e2da1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab1000 msgr2=0x559e2da1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab1000 0x559e2da1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dbb6400 msgr2=0x559e2dc16580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dbb6400 0x559e2dc16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2cc87a00 con 0x559e2dab1400 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39e43d700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559e2dc1a240 tx=0x559e2d9f4ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c5e7e444e06698 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39542b700 1 -- 172.21.15.98:0/4003104471 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9b080 con 0x559e2dab1400 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39542b700 1 -- 172.21.15.98:0/4003104471 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e2da44780 con 0x559e2dab1400 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39542b700 1 -- 172.21.15.98:0/4003104471 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559e2cc9a6e0 con 0x559e2dab1400 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e2cc86000 con 0x559e2dab1400 2023-10-06T01:41:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559e2db0d040 con 0x559e2dab1400 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 4 waiting for MDS map... 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39542b700 1 -- 172.21.15.98:0/4003104471 <== mon.0 v2:172.21.15.98:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x559e2cca68c0 con 0x559e2dab1400 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff39542b700 1 -- 172.21.15.98:0/4003104471 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 3663+0+0 (secure 0 0 0) 0x559e2dc20600 con 0x559e2dab1400 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 4 Got MDS map 23 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x559e2dbb6800 0x559e2dc16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.337+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:1 8.f 8:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dbb6c00 con 0x559e2dbb6800 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.338+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x559e2dbb6800 0x559e2dc16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.338+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x559e2dbb6800 0x559e2dc16b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=1c2d698762213639 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.339+0000 7ff39ec3e700 1 -- 172.21.15.98:0/4003104471 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v44'1 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41200 con 0x559e2dbb6800 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.339+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x559e2dbb7000 0x559e2dc17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.339+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:2 8.14 8:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dbb7400 con 0x559e2dbb7000 2023-10-06T01:41:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.339+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x559e2dbb7000 0x559e2dc17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.340+0000 7ff39dc3c700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x559e2dbb7000 0x559e2dc17080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=547b221bee0d8d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4003104471 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v44'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41440 con 0x559e2dbb7000 2023-10-06T01:41:15.353 INFO:teuthology.orchestra.run.smithi098.stdout:writing EResetJournal entry 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:3 8.14 8:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dbb7800 con 0x559e2dbb7000 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:4 8.1b 8:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dbb7c00 con 0x559e2dbb7000 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:5 8.b 8:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dc1e800 con 0x559e2dbb6800 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x559e2dc1ec00 0x559e2dc17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:6 8.1f 8:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dc1f000 con 0x559e2dc1ec00 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- osd_op(unknown.0.0:7 8.1a 8:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dc1f400 con 0x559e2dc1ec00 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- osd_op(unknown.0.0:8 8.1e 8:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dc1f800 con 0x559e2dbb6800 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x559e2dc1ec00 0x559e2dc17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.358 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.341+0000 7ff39ec3e700 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x559e2dc1ec00 0x559e2dc17600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=2ae85266c952f470 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.342+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4003104471 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== osd_op_reply(3 200.00000001 [delete] v44'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41440 con 0x559e2dbb7000 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.344+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4003104471 <== osd.4 v2:172.21.15.184:6824/120887 3 ==== osd_op_reply(4 200.00000002 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41440 con 0x559e2dbb7000 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.344+0000 7ff393c28700 1 -- 172.21.15.98:0/4003104471 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- osd_op(unknown.0.0:9 8.14 8:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x559e2dc1e800 con 0x559e2dbb7000 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.344+0000 7ff39ec3e700 1 -- 172.21.15.98:0/4003104471 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== osd_op_reply(8 200.00000006 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41200 con 0x559e2dbb6800 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.347+0000 7ff39ec3e700 1 -- 172.21.15.98:0/4003104471 <== osd.3 v2:172.21.15.98:6809/120655 3 ==== osd_op_reply(5 200.00000003 [delete] v44'2 uv1 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41200 con 0x559e2dbb6800 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.347+0000 7ff39ec3e700 1 -- 172.21.15.98:0/4003104471 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== osd_op_reply(7 200.00000005 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41200 con 0x559e2dc1ec00 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.347+0000 7ff39ec3e700 1 -- 172.21.15.98:0/4003104471 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== osd_op_reply(6 200.00000004 [delete] v44'5 uv4 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41200 con 0x559e2dc1ec00 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4003104471 <== osd.4 v2:172.21.15.184:6824/120887 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v44'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x559e2da41440 con 0x559e2dbb7000 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 4 Successfully wrote new journal pointer and header for rank 3:0 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x559e2dc1ec00 msgr2=0x559e2dc17600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x559e2dc1ec00 0x559e2dc17600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x559e2dbb6800 msgr2=0x559e2dc16b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x559e2dbb6800 0x559e2dc16b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x559e2dbb7000 msgr2=0x559e2dc17080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x559e2dbb7000 0x559e2dc17080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff39dc3c700 1 -- 172.21.15.98:0/4003104471 reap_dead start 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 msgr2=0x559e2dc16000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559e2dc1a240 tx=0x559e2d9f4ae0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 shutdown_connections 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/4003104471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab1400 0x559e2dc16000 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 >> 172.21.15.98:0/4003104471 conn(0x559e2dab0c00 msgr2=0x559e2d9ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 shutdown_connections 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/4003104471 wait complete. 2023-10-06T01:41:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 msgr2=0x559e2da1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 0x559e2da1d700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x559e2ccf1e00 tx=0x559e2d9f5980 comp rx=0 tx=0).stop 2023-10-06T01:41:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 msgr2=0x559e2da1cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x559e2dada060 tx=0x559e2da520c0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 shutdown_connections 2023-10-06T01:41:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x559e2dbb6000 0x559e2da1d700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2dab0800 0x559e2da1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2dab0000 0x559e2da1cc00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/436502718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2dab0400 0x559e2da1c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 >> 172.21.15.98:0/436502718 conn(0x559e2da23000 msgr2=0x559e2d9eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 shutdown_connections 2023-10-06T01:41:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/436502718 wait complete. 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 msgr2=0x559e2da1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559e2d9e0060 tx=0x559e2da521e0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 shutdown_connections 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e2da22c00 0x559e2da1b080 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x559e2da22400 0x559e2da1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 --2- 172.21.15.98:0/567949634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e2da22800 0x559e2da1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.352+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 >> 172.21.15.98:0/567949634 conn(0x559e2da22000 msgr2=0x559e2d9eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.353+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 shutdown_connections 2023-10-06T01:41:15.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.353+0000 7ff3a4f6f240 1 -- 172.21.15.98:0/567949634 wait complete. 2023-10-06T01:41:15.365 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'journal', 'reset', '--force'] in time 0:00:00.106930, result: writing EResetJournal entry 2023-10-06T01:41:15.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs joinable true 2023-10-06T01:41:15.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.514+0000 7f7bcf7f7700 1 Processor -- start 2023-10-06T01:41:15.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.514+0000 7f7bcf7f7700 1 -- start start 2023-10-06T01:41:15.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcf7f7700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bc813fe40 0x7f7bc80c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcf7f7700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bc806e380 0x7f7bc815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcf7f7700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc806e820 con 0x7f7bc80c7b00 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcf7f7700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc815ae20 con 0x7f7bc806e380 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcf7f7700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc806ea60 con 0x7f7bc813fe40 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bc806e380 0x7f7bc815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bc806e380 0x7f7bc815a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36076/0 (socket says 172.21.15.98:36076) 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 -- 172.21.15.98:0/3795358682 learned_addr learned my addr 172.21.15.98:0/3795358682 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bccd92700 1 --2- 172.21.15.98:0/3795358682 >> v2:172.21.15.184:3300/0 conn(0x7f7bc813fe40 0x7f7bc80c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 -- 172.21.15.98:0/3795358682 >> v2:172.21.15.184:3300/0 conn(0x7f7bc813fe40 msgr2=0x7f7bc80c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 --2- 172.21.15.98:0/3795358682 >> v2:172.21.15.184:3300/0 conn(0x7f7bc813fe40 0x7f7bc80c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 -- 172.21.15.98:0/3795358682 >> v1:172.21.15.184:6789/0 conn(0x7f7bc80c7b00 legacy=0x7f7bc813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 -- 172.21.15.98:0/3795358682 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc806ed40 con 0x7f7bc806e380 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bbffff700 1 -- 172.21.15.98:0/3795358682 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301541826 0 0) 0x7f7bc4001110 con 0x7f7bc80c7b00 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bcdd94700 1 --2- 172.21.15.98:0/3795358682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc815a8e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000d1a0 tx=0x7f7bc000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e50918dfdc61dcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.515+0000 7f7bbffff700 1 -- 172.21.15.98:0/3795358682 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0010070 con 0x7f7bc806e380 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bbffff700 1 -- 172.21.15.98:0/3795358682 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc000ddd0 con 0x7f7bc806e380 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bbffff700 1 -- 172.21.15.98:0/3795358682 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000b320 con 0x7f7bc806e380 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/3795358682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 msgr2=0x7f7bc815a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/3795358682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc815a8e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000d1a0 tx=0x7f7bc000ba50 comp rx=0 tx=0).stop 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/3795358682 shutdown_connections 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/3795358682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc815a8e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/3795358682 >> v2:172.21.15.184:3300/0 conn(0x7f7bc813fe40 0x7f7bc80c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/3795358682 >> 172.21.15.98:0/3795358682 conn(0x7f7bc80cc0e0 msgr2=0x7f7bc80cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:15.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/3795358682 shutdown_connections 2023-10-06T01:41:15.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.516+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/3795358682 wait complete. 2023-10-06T01:41:15.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.517+0000 7f7bcf7f7700 1 Processor -- start 2023-10-06T01:41:15.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.517+0000 7f7bcf7f7700 1 -- start start 2023-10-06T01:41:15.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc80c7b00 0x7f7bc813b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc813fe40 0x7f7bc813bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80b09b0 con 0x7f7bc806e380 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc813c330 con 0x7f7bc80c7b00 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcf7f7700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80afe30 con 0x7f7bc813fe40 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36090/0 (socket says 172.21.15.98:36090) 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 -- 172.21.15.98:0/530677607 learned_addr learned my addr 172.21.15.98:0/530677607 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bccd92700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc80c7b00 0x7f7bc813b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcdd94700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc813fe40 0x7f7bc813bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 -- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc813fe40 msgr2=0x7f7bc813bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.524 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc813fe40 0x7f7bc813bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 -- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc80c7b00 msgr2=0x7f7bc813b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc80c7b00 0x7f7bc813b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.518+0000 7f7bcd593700 1 -- 172.21.15.98:0/530677607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc000bcf0 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bcd593700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c7e0 tx=0x7f7bc4007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4011590 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4011710 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc80b0110 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4011ad0 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.519+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc80b06d0 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.520+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc8064ad0 con 0x7f7bc806e380 2023-10-06T01:41:15.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.520+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bc401b3b0 con 0x7f7bc806e380 2023-10-06T01:41:15.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.520+0000 7f7bbd7fa700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:15.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.520+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f7bc4083d80 con 0x7f7bc806e380 2023-10-06T01:41:15.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.520+0000 7f7bccd92700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:15.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.521+0000 7f7bccd92700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 0x7f7bac0693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7bc813c920 tx=0x7f7bb80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:15.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.524+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc4050b30 con 0x7f7bc806e380 2023-10-06T01:41:15.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:15.750+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f7bc806e820 con 0x7f7bc806e380 2023-10-06T01:41:16.027 INFO:tasks.ceph.mon.a.smithi098.stderr:2023-10-06T01:41:16.026+0000 7f8a67e63700 -1 log_channel(cluster) log [ERR] : Health check update: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T01:41:16.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.028+0000 7f7bbd7fa700 1 -- 172.21.15.98:0/530677607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"}]=0 recovery_fs marked joinable; MDS may join as newly active. v24) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7bc806e820 con 0x7f7bc806e380 2023-10-06T01:41:16.031 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:16.030+0000 7fb8b85ba700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T01:41:16.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 msgr2=0x7f7bac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 0x7f7bac0693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7bc813c920 tx=0x7f7bb80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 msgr2=0x7f7bc806a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c7e0 tx=0x7f7bc4007c60 comp rx=0 tx=0).stop 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 shutdown_connections 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7bac066f30 0x7f7bac0693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc813fe40 0x7f7bc813bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bc806e380 0x7f7bc806a1f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 --2- 172.21.15.98:0/530677607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc80c7b00 0x7f7bc813b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 >> 172.21.15.98:0/530677607 conn(0x7f7bc80cc0e0 msgr2=0x7f7bc813e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.031+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 shutdown_connections 2023-10-06T01:41:16.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.032+0000 7f7bcf7f7700 1 -- 172.21.15.98:0/530677607 wait complete. 2023-10-06T01:41:16.035 INFO:teuthology.orchestra.run.smithi098.stderr:recovery_fs marked joinable; MDS may join as newly active. 2023-10-06T01:41:16.046 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:16.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.160+0000 7fec8bca3700 1 Processor -- start 2023-10-06T01:41:16.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.160+0000 7fec8bca3700 1 -- start start 2023-10-06T01:41:16.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8bca3700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec841595a0 0x7fec840aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8bca3700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 0x7fec840aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8bca3700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840b0ca0 con 0x7fec840b07a0 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8bca3700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840b0f80 con 0x7fec841595a0 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8bca3700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840b1260 con 0x7fec840b49e0 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec89a3f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec841595a0 0x7fec840aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8a240700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fec840b07a0 0x7fec840b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52400/0 (socket says 172.21.15.98:52400) 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8a240700 1 -- 172.21.15.98:0/2561389253 learned_addr learned my addr 172.21.15.98:0/2561389253 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.161+0000 7fec8923e700 1 --2- 172.21.15.98:0/2561389253 >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 0x7fec840aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec89a3f700 1 -- 172.21.15.98:0/2561389253 >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 msgr2=0x7fec840aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec89a3f700 1 --2- 172.21.15.98:0/2561389253 >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 0x7fec840aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec89a3f700 1 -- 172.21.15.98:0/2561389253 >> v1:172.21.15.98:6789/0 conn(0x7fec840b07a0 legacy=0x7fec840b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec89a3f700 1 -- 172.21.15.98:0/2561389253 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840b1540 con 0x7fec841595a0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec88a3d700 1 -- 172.21.15.98:0/2561389253 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784599775 0 0) 0x7fec840b0ca0 con 0x7fec840b07a0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8923e700 1 --2- 172.21.15.98:0/2561389253 >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 0x7fec840aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec89a3f700 1 --2- 172.21.15.98:0/2561389253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec841595a0 0x7fec840aa570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9ea8db1447f3b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec88a3d700 1 -- 172.21.15.98:0/2561389253 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8000e070 con 0x7fec841595a0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec88a3d700 1 -- 172.21.15.98:0/2561389253 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8000bbf0 con 0x7fec841595a0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec88a3d700 1 -- 172.21.15.98:0/2561389253 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80012430 con 0x7fec841595a0 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 -- 172.21.15.98:0/2561389253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec841595a0 msgr2=0x7fec840aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 --2- 172.21.15.98:0/2561389253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec841595a0 0x7fec840aa570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80009780 comp rx=0 tx=0).stop 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 -- 172.21.15.98:0/2561389253 shutdown_connections 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 --2- 172.21.15.98:0/2561389253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec841595a0 0x7fec840aa570 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 --2- 172.21.15.98:0/2561389253 >> v2:172.21.15.184:3300/0 conn(0x7fec840b49e0 0x7fec840aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 -- 172.21.15.98:0/2561389253 >> 172.21.15.98:0/2561389253 conn(0x7fec84099830 msgr2=0x7fec8406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:16.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.162+0000 7fec8bca3700 1 -- 172.21.15.98:0/2561389253 shutdown_connections 2023-10-06T01:41:16.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.163+0000 7fec8bca3700 1 -- 172.21.15.98:0/2561389253 wait complete. 2023-10-06T01:41:16.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 Processor -- start 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 -- start start 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 0x7fec8416a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec841595a0 0x7fec8409d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84169e50 con 0x7fec840b49e0 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8409db60 con 0x7fec840b07a0 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8bca3700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8409de40 con 0x7fec841595a0 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8923e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8923e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36106/0 (socket says 172.21.15.98:36106) 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.164+0000 7fec8923e700 1 -- 172.21.15.98:0/1997927315 learned_addr learned my addr 172.21.15.98:0/1997927315 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec89a3f700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 0x7fec8416a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8a240700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec841595a0 0x7fec8409d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 -- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec841595a0 msgr2=0x7fec8409d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec841595a0 0x7fec8409d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 -- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 msgr2=0x7fec8416a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 0x7fec8416a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 -- 172.21.15.98:0/1997927315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80009a20 con 0x7fec840b49e0 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec89a3f700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 0x7fec8416a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8923e700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c800 tx=0x7fec74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:16.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74011600 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec74011760 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec84169ae0 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74011b20 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.165+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec84169780 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.166+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec84169c70 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.167+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec740073d0 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.167+0000 7fec79ffb700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.167+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fec74084aa0 con 0x7fec840b49e0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.167+0000 7fec89a3f700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:16.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.167+0000 7fec89a3f700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:16.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.171+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec74051850 con 0x7fec840b49e0 2023-10-06T01:41:16.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.397+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec84064ad0 con 0x7fec840b49e0 2023-10-06T01:41:16.399 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.397+0000 7fec79ffb700 1 -- 172.21.15.98:0/1997927315 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5668 (secure 0 0 0) 0x7fec74054e70 con 0x7fec840b49e0 2023-10-06T01:41:16.399 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:16.400 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:16.027390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:16.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 msgr2=0x7fec680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80017000 comp rx=0 tx=0).stop 2023-10-06T01:41:16.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 msgr2=0x7fec841560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:16.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c800 tx=0x7fec74008040 comp rx=0 tx=0).stop 2023-10-06T01:41:16.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 shutdown_connections 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec841595a0 0x7fec8409d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec840b07a0 0x7fec8416a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 --2- 172.21.15.98:0/1997927315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec840b49e0 0x7fec841560f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 >> 172.21.15.98:0/1997927315 conn(0x7fec84099830 msgr2=0x7fec84068400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 shutdown_connections 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:16.399+0000 7fec8bca3700 1 -- 172.21.15.98:0/1997927315 wait complete. 2023-10-06T01:41:16.402 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2023-10-06T01:41:16.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:41:04.761696+0000', 'modified': '2023-10-06T01:41:16.027390+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5409': {'gid': 5409, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.98:6837/1433484145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1433484145}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1433484145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T01:41:16.815 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5409:up:replay 2023-10-06T01:41:17.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:17.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.940+0000 7f61c9d81700 1 Processor -- start 2023-10-06T01:41:17.942 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 -- start start 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f61c40cb630 0x7f61c40691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61c415a220 0x7f61c40ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c4068d50 con 0x7f61c4069720 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c4154060 con 0x7f61c40cb630 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c9d81700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c415a6b0 con 0x7f61c415a220 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.941+0000 7f61c37fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f61c40cb630 0x7f61c40691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f61c40cb630 0x7f61c40691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36112/0 (socket says 172.21.15.98:36112) 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 -- 172.21.15.98:0/2956258397 learned_addr learned my addr 172.21.15.98:0/2956258397 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:17.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c3fff700 1 --2- 172.21.15.98:0/2956258397 >> v2:172.21.15.184:3300/0 conn(0x7f61c415a220 0x7f61c40ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 -- 172.21.15.98:0/2956258397 >> v2:172.21.15.184:3300/0 conn(0x7f61c415a220 msgr2=0x7f61c40ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 --2- 172.21.15.98:0/2956258397 >> v2:172.21.15.184:3300/0 conn(0x7f61c415a220 0x7f61c40ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 -- 172.21.15.98:0/2956258397 >> v1:172.21.15.184:6789/0 conn(0x7f61c4069720 legacy=0x7f61c4159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 -- 172.21.15.98:0/2956258397 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c415a990 con 0x7f61c40cb630 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c37fe700 1 --2- 172.21.15.98:0/2956258397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c40cb630 0x7f61c40691e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f61c4068cc0 tx=0x7f61b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8389af397ab2539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c27fc700 1 -- 172.21.15.98:0/2956258397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b400e070 con 0x7f61c40cb630 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c27fc700 1 -- 172.21.15.98:0/2956258397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b400ba10 con 0x7f61c40cb630 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.942+0000 7f61c27fc700 1 -- 172.21.15.98:0/2956258397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40092e0 con 0x7f61c40cb630 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 -- 172.21.15.98:0/2956258397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c40cb630 msgr2=0x7f61c40691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:17.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2956258397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c40cb630 0x7f61c40691e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f61c4068cc0 tx=0x7f61b4009840 comp rx=0 tx=0).stop 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 -- 172.21.15.98:0/2956258397 shutdown_connections 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2956258397 >> v2:172.21.15.184:3300/0 conn(0x7f61c415a220 0x7f61c40ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2956258397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c40cb630 0x7f61c40691e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 -- 172.21.15.98:0/2956258397 >> 172.21.15.98:0/2956258397 conn(0x7f61c4070810 msgr2=0x7f61c4070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 -- 172.21.15.98:0/2956258397 shutdown_connections 2023-10-06T01:41:17.947 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.943+0000 7f61c9d81700 1 -- 172.21.15.98:0/2956258397 wait complete. 2023-10-06T01:41:17.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.944+0000 7f61c9d81700 1 Processor -- start 2023-10-06T01:41:17.949 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.944+0000 7f61c9d81700 1 -- start start 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c40cb630 0x7f61c4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c415a220 0x7f61c41562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4154a20 con 0x7f61c4069720 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4153a50 con 0x7f61c40cb630 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4154740 con 0x7f61c415a220 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36128/0 (socket says 172.21.15.98:36128) 2023-10-06T01:41:17.950 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 -- 172.21.15.98:0/2674259045 learned_addr learned my addr 172.21.15.98:0/2674259045 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 -- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c415a220 msgr2=0x7f61c41562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c415a220 0x7f61c41562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 -- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c40cb630 msgr2=0x7f61c4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c40cb630 0x7f61c4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 -- 172.21.15.98:0/2674259045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009ae0 con 0x7f61c4069720 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c37fe700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f61c40cdfe0 tx=0x7f61b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4012b30 con 0x7f61c4069720 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4012cb0 con 0x7f61c4069720 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b401d670 con 0x7f61c4069720 2023-10-06T01:41:17.951 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.945+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c40c34f0 con 0x7f61c4069720 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.946+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40c3ab0 con 0x7f61c4069720 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.946+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c4064ad0 con 0x7f61c4069720 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.947+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61b401da60 con 0x7f61c4069720 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.947+0000 7f61ab7fe700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 0x7f61b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.947+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f61b4086150 con 0x7f61c4069720 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.947+0000 7f61c2ffd700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 0x7f61b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:17.952 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.948+0000 7f61c2ffd700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 0x7f61b00693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f61c415c540 tx=0x7f61b8006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:17.953 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:17.952+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61b4052f00 con 0x7f61c4069720 2023-10-06T01:41:18.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.177+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61c404bb80 con 0x7f61c4069720 2023-10-06T01:41:18.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.178+0000 7f61ab7fe700 1 -- 172.21.15.98:0/2674259045 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f61b4056520 con 0x7f61c4069720 2023-10-06T01:41:18.179 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:18.180 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:18.033315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:18.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 msgr2=0x7f61b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:18.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 0x7f61b00693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f61c415c540 tx=0x7f61b8006c60 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 msgr2=0x7f61c40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f61c40cdfe0 tx=0x7f61b4017000 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 shutdown_connections 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f61b0066f30 0x7f61b00693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61c415a220 0x7f61c41562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61c4069720 0x7f61c40b2090 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 --2- 172.21.15.98:0/2674259045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61c40cb630 0x7f61c4152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 >> 172.21.15.98:0/2674259045 conn(0x7f61c4070810 msgr2=0x7f61c4069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 shutdown_connections 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:18.180+0000 7f61c9d81700 1 -- 172.21.15.98:0/2674259045 wait complete. 2023-10-06T01:41:18.183 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 26 2023-10-06T01:41:18.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:41:04.761696+0000', 'modified': '2023-10-06T01:41:18.033315+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5409': {'gid': 5409, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.98:6837/1433484145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1433484145}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1433484145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T01:41:18.596 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5409:up:rejoin 2023-10-06T01:41:19.597 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:19.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.720+0000 7f3f255cf700 1 Processor -- start 2023-10-06T01:41:19.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.720+0000 7f3f255cf700 1 -- start start 2023-10-06T01:41:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f255cf700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 0x7f3f200aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f255cf700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 0x7f3f200b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f255cf700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f200b0ca0 con 0x7f3f200b49e0 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f255cf700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f200b0f80 con 0x7f3f200b07a0 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f255cf700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200b1260 con 0x7f3f201595a0 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1f7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 0x7f3f200b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f167fc700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3f200b49e0 0x7f3f200aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:34284/0 (socket says 172.21.15.98:34284) 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f167fc700 1 -- 172.21.15.98:0/3912134366 learned_addr learned my addr 172.21.15.98:0/3912134366 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 --2- 172.21.15.98:0/3912134366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 0x7f3f200aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 -- 172.21.15.98:0/3912134366 >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 msgr2=0x7f3f200b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 --2- 172.21.15.98:0/3912134366 >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 0x7f3f200b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 -- 172.21.15.98:0/3912134366 >> v1:172.21.15.184:6789/0 conn(0x7f3f200b49e0 legacy=0x7f3f200aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 -- 172.21.15.98:0/3912134366 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200b1540 con 0x7f3f201595a0 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1f7fe700 1 --2- 172.21.15.98:0/3912134366 >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 0x7f3f200b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1effd700 1 --2- 172.21.15.98:0/3912134366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 0x7f3f200aa570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f100098d0 tx=0x7f3f10009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6aa50255441d86f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1e7fc700 1 -- 172.21.15.98:0/3912134366 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10005c10 con 0x7f3f201595a0 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1e7fc700 1 -- 172.21.15.98:0/3912134366 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10005d70 con 0x7f3f201595a0 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.722+0000 7f3f1e7fc700 1 -- 172.21.15.98:0/3912134366 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10006280 con 0x7f3f201595a0 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 -- 172.21.15.98:0/3912134366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 msgr2=0x7f3f200aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 --2- 172.21.15.98:0/3912134366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 0x7f3f200aa570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f100098d0 tx=0x7f3f10009120 comp rx=0 tx=0).stop 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 -- 172.21.15.98:0/3912134366 shutdown_connections 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 --2- 172.21.15.98:0/3912134366 >> v2:172.21.15.184:3300/0 conn(0x7f3f200b07a0 0x7f3f200b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 --2- 172.21.15.98:0/3912134366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f201595a0 0x7f3f200aa570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 -- 172.21.15.98:0/3912134366 >> 172.21.15.98:0/3912134366 conn(0x7f3f20099830 msgr2=0x7f3f2006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:19.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 -- 172.21.15.98:0/3912134366 shutdown_connections 2023-10-06T01:41:19.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.723+0000 7f3f255cf700 1 -- 172.21.15.98:0/3912134366 wait complete. 2023-10-06T01:41:19.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.724+0000 7f3f255cf700 1 Processor -- start 2023-10-06T01:41:19.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.724+0000 7f3f255cf700 1 -- start start 2023-10-06T01:41:19.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f200b07a0 0x7f3f2009d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 0x7f3f20146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20145570 con 0x7f3f200b49e0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20144960 con 0x7f3f201595a0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f255cf700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20144c40 con 0x7f3f200b07a0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36134/0 (socket says 172.21.15.98:36134) 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 -- 172.21.15.98:0/2222113679 learned_addr learned my addr 172.21.15.98:0/2222113679 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f1effd700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f200b07a0 0x7f3f2009d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f1f7fe700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 0x7f3f20146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 -- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f200b07a0 msgr2=0x7f3f2009d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f200b07a0 0x7f3f2009d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 -- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 msgr2=0x7f3f20146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 0x7f3f20146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 -- 172.21.15.98:0/2222113679 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f18009a20 con 0x7f3f200b49e0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f1f7fe700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 0x7f3f20146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.725+0000 7f3f167fc700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800c790 tx=0x7f3f08007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08005b30 con 0x7f3f200b49e0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08005c90 con 0x7f3f200b49e0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f100093c0 con 0x7f3f200b49e0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08004a80 con 0x7f3f200b49e0 2023-10-06T01:41:19.731 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f20145200 con 0x7f3f200b49e0 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.726+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f20145720 con 0x7f3f200b49e0 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.727+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f080070d0 con 0x7f3f200b49e0 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.727+0000 7f3f177fe700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 0x7f3f000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.727+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f3f08083d80 con 0x7f3f200b49e0 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.727+0000 7f3f1effd700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 0x7f3f000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:19.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.728+0000 7f3f1effd700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 0x7f3f000693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f20157580 tx=0x7f3f10016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:19.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.732+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f0804b5a0 con 0x7f3f200b49e0 2023-10-06T01:41:19.958 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.956+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f20064ad0 con 0x7f3f200b49e0 2023-10-06T01:41:19.959 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.957+0000 7f3f177fe700 1 -- 172.21.15.98:0/2222113679 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f3f20145720 con 0x7f3f200b49e0 2023-10-06T01:41:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:19.959 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:19.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 msgr2=0x7f3f000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 0x7f3f000693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f20157580 tx=0x7f3f10016000 comp rx=0 tx=0).stop 2023-10-06T01:41:19.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 msgr2=0x7f3f201697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:19.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800c790 tx=0x7f3f08007f40 comp rx=0 tx=0).stop 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 shutdown_connections 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3f00066f30 0x7f3f000693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f201595a0 0x7f3f20146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f200b07a0 0x7f3f2009d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 --2- 172.21.15.98:0/2222113679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f200b49e0 0x7f3f201697d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 >> 172.21.15.98:0/2222113679 conn(0x7f3f20099830 msgr2=0x7f3f20069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 shutdown_connections 2023-10-06T01:41:19.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:19.959+0000 7f3f255cf700 1 -- 172.21.15.98:0/2222113679 wait complete. 2023-10-06T01:41:19.963 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:20.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T01:41:04.761696+0000', 'modified': '2023-10-06T01:41:19.034849+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5409': {'gid': 5409, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.98:6837/1433484145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1433484145}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1433484145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T01:41:20.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T01:41:20.376 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T01:41:20.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:41:20.442 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:41:20.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 Processor -- start 2023-10-06T01:41:20.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 -- start start 2023-10-06T01:41:20.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70480cadc0 0x7f7048159f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7048068d50 con 0x7f7048159b10 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704815a6b0 con 0x7f70480cb630 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704e932700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704815a4d0 con 0x7f70480cadc0 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f70477fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7048159b10 0x7f70480ca6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52410/0 (socket says 172.21.15.98:52410) 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f70477fe700 1 -- 172.21.15.98:0/2383216260 learned_addr learned my addr 172.21.15.98:0/2383216260 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239227375 0 0) 0x7f7048068d50 con 0x7f7048159b10 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7030003570 con 0x7f7048159b10 2023-10-06T01:41:20.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f704cecf700 1 --2- 172.21.15.98:0/2383216260 >> v2:172.21.15.184:3300/0 conn(0x7f70480cadc0 0x7f7048159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840701765 0 0) 0x7f7030003570 con 0x7f7048159b10 2023-10-06T01:41:20.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 >> v2:172.21.15.184:3300/0 conn(0x7f70480cadc0 msgr2=0x7f7048159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 --2- 172.21.15.98:0/2383216260 >> v2:172.21.15.184:3300/0 conn(0x7f70480cadc0 0x7f7048159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 >> v1:172.21.15.184:6789/0 conn(0x7f70480cb630 legacy=0x7f70480691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.499 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704815a990 con 0x7f7048159b10 2023-10-06T01:41:20.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f7038001470 con 0x7f7048159b10 2023-10-06T01:41:20.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.496+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 >> v1:172.21.15.98:6789/0 conn(0x7f7048159b10 legacy=0x7f70480ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7030003a60 0x7f7030003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030004810 0x7f7030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f703000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704815a4d0 con 0x7f70300051c0 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703000c0d0 con 0x7f7030004810 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703000c290 con 0x7f7030003a60 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f703000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7047fff700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7030003a60 0x7f7030003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f70477fe700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030004810 0x7f7030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 -- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7030003a60 msgr2=0x7f7030003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7030003a60 0x7f7030003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 -- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030004810 msgr2=0x7f7030004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030004810 0x7f7030004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 -- 172.21.15.98:0/2383216260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7030003570 con 0x7f70300051c0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 -- 172.21.15.98:0/2383216260 reap_dead start 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f704cecf700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f703000ba60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f704000e210 tx=0x7f704000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aba288a3c97cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000eac0 con 0x7f70300051c0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000f040 con 0x7f70300051c0 2023-10-06T01:41:20.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.497+0000 7f7046ffd700 1 -- 172.21.15.98:0/2383216260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000ee70 con 0x7f70300051c0 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 -- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 msgr2=0x7f703000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f703000ba60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f704000e210 tx=0x7f704000b040 comp rx=0 tx=0).stop 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 -- 172.21.15.98:0/2383216260 shutdown_connections 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 --2- 172.21.15.98:0/2383216260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f703000ba60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 -- 172.21.15.98:0/2383216260 >> 172.21.15.98:0/2383216260 conn(0x7f7048070810 msgr2=0x7f7048070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 -- 172.21.15.98:0/2383216260 shutdown_connections 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.498+0000 7f704e932700 1 -- 172.21.15.98:0/2383216260 wait complete. 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.499+0000 7f704e932700 1 Processor -- start 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.499+0000 7f704e932700 1 -- start start 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70480ce7b0 0x7f70481555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7048155b10 0x7f70481472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048156a30 con 0x7f70300051c0 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048155f90 con 0x7f70480ce7b0 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704e932700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480c5550 con 0x7f7048155b10 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36162/0 (socket says 172.21.15.98:36162) 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 -- 172.21.15.98:0/1044818312 learned_addr learned my addr 172.21.15.98:0/1044818312 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f70477fe700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70480ce7b0 0x7f70481555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f704cecf700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7048155b10 0x7f70481472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 -- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7048155b10 msgr2=0x7f70481472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7048155b10 0x7f70481472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 -- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70480ce7b0 msgr2=0x7f70481555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70480ce7b0 0x7f70481555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 -- 172.21.15.98:0/1044818312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048156190 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f7047fff700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f703c002980 tx=0x7f703c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.500+0000 7f702ffff700 1 -- 172.21.15.98:0/1044818312 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c00f8d0 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.501+0000 7f702ffff700 1 -- 172.21.15.98:0/1044818312 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c00fa30 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.501+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048156470 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.501+0000 7f702ffff700 1 -- 172.21.15.98:0/1044818312 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c00fdd0 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.501+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480b2290 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.502+0000 7f702ffff700 1 -- 172.21.15.98:0/1044818312 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f703c013600 con 0x7f70300051c0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.502+0000 7f702ffff700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 0x7f70340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.502+0000 7f70477fe700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 0x7f70340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.503+0000 7f702ffff700 1 -- 172.21.15.98:0/1044818312 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f703c084e30 con 0x7f70300051c0 2023-10-06T01:41:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.503+0000 7f70477fe700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 0x7f70340693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70480cebc0 tx=0x7f7038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.504+0000 7f7057c2ab80 1 Processor -- start 2023-10-06T01:41:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.504+0000 7f7057c2ab80 1 -- start start 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf0d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcf0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce14a70 con 0x55a8bcebd840 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce12270 con 0x55a8bcf0dc00 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce0e060 con 0x55a8bcebecc0 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36170/0 (socket says 172.21.15.98:36170) 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.505+0000 7f7047fff700 1 -- 172.21.15.98:0/1288174652 learned_addr learned my addr 172.21.15.98:0/1288174652 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f70477fe700 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf0d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f704cecf700 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcf0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 -- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 msgr2=0x55a8bcf0d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf0d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 -- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 msgr2=0x55a8bcf0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcf0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 -- 172.21.15.98:0/1288174652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bcca8270 con 0x55a8bcebd840 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f7047fff700 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f703c082ec0 tx=0x7f703c082df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7e1ff6f0d2ff14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f702d7fa700 1 -- 172.21.15.98:0/1288174652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c055950 con 0x55a8bcebd840 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f702d7fa700 1 -- 172.21.15.98:0/1288174652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c058af0 con 0x55a8bcebd840 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.506+0000 7f702d7fa700 1 -- 172.21.15.98:0/1288174652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c05c840 con 0x55a8bcebd840 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- 172.21.15.98:0/1288174652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 msgr2=0x55a8bcebdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f703c082ec0 tx=0x7f703c082df0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- 172.21.15.98:0/1288174652 shutdown_connections 2023-10-06T01:41:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcf0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcebdc10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/1288174652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf0d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- 172.21.15.98:0/1288174652 >> 172.21.15.98:0/1288174652 conn(0x55a8bcebf160 msgr2=0x55a8bcebf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- 172.21.15.98:0/1288174652 shutdown_connections 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- 172.21.15.98:0/1288174652 wait complete. 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 -- start start 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.507+0000 7f7057c2ab80 1 Processor -- start 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bceec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcdb9890 con 0x55a8bcebd840 2023-10-06T01:41:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcdd4ea0 con 0x55a8bcf0dc00 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcc90e30 con 0x55a8bcebecc0 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7057c2ab80 10 client.0 _ll_register_callbacks cb 0x55a8bce80e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36176/0 (socket says 172.21.15.98:36176) 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 -- 172.21.15.98:0/3080141965 learned_addr learned my addr 172.21.15.98:0/3080141965 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f70477fe700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bceec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f704cecf700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 -- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 msgr2=0x55a8bceec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bceec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 -- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 msgr2=0x55a8bcee6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.508+0000 7f7047fff700 1 -- 172.21.15.98:0/3080141965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480b2290 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7047fff700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f703c082f70 tx=0x7f703c06c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce50dcc7047feae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c082070 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f703c06a070 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8bceb2940 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f703c055950 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 10 client.5700 fetch_fsmap learned FSMap version 27 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55a8bcc4afe0 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8bceb2940 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f703c084070 con 0x55a8bcebd840 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 10 client.5700 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T01:41:20.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 10 client.5700 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 20 client.5700 populate_metadata read hostname 'smithi098' 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcee87e0 0x55a8bcee8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 4 client.5700 mds_command: new command op to 5409 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 --> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8bce8f870 con 0x55a8bcee87e0 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.509+0000 7f70477fe700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcee87e0 0x55a8bcee8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f70267fc700 10 client.5700 ms_handle_connect on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f70477fe700 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcee87e0 0x55a8bcee8bb0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f8aabf4742c3ec server_cookie=bcb09a4d6a0511a4 in_seq=0 out_seq=0 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f70267fc700 1 -- 172.21.15.98:0/3080141965 <== mds.0 v2:172.21.15.98:6836/1433484145 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a8bce8f870 con 0x55a8bcee87e0 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f70267fc700 10 client.5700 handle_command_reply: tid=0 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 client.5700 shutdown 2023-10-06T01:41:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 msgr2=0x55a8bcefbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f703c082f70 tx=0x7f703c06c000 comp rx=0 tx=0).stop 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 shutdown_connections 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcee87e0 0x55a8bcee8bb0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcebd840 0x55a8bcefbac0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f70267fc700 0 client.5700 ms_handle_reset on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/3080141965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bceec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 >> 172.21.15.98:0/3080141965 conn(0x55a8bcebf160 msgr2=0x55a8bceb8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 shutdown_connections 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.510+0000 7f7057c2ab80 1 -- 172.21.15.98:0/3080141965 wait complete. 2023-10-06T01:41:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.511+0000 7f7057c2ab80 20 client.5700 trim_cache size 0 max 16384 2023-10-06T01:41:20.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.527+0000 7f7057c2ab80 1 Processor -- start 2023-10-06T01:41:20.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.527+0000 7f7057c2ab80 1 -- start start 2023-10-06T01:41:20.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcebe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcee8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcebea50 con 0x55a8bcf0dc00 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcee5c10 con 0x55a8bcebd840 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bceb8370 con 0x55a8bcebecc0 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36180/0 (socket says 172.21.15.98:36180) 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 -- 172.21.15.98:0/2870403578 learned_addr learned my addr 172.21.15.98:0/2870403578 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 -- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 msgr2=0x55a8bcee8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f70477fe700 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcee8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f7047fff700 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcebe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcee8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 -- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 msgr2=0x55a8bcebe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcebe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 -- 172.21.15.98:0/2870403578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bcddc780 con 0x55a8bcf0dc00 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.528+0000 7f704cecf700 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7040009750 tx=0x7f7040019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa825eae00252463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f702dffb700 1 -- 172.21.15.98:0/2870403578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040023040 con 0x55a8bcf0dc00 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f702dffb700 1 -- 172.21.15.98:0/2870403578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000eac0 con 0x55a8bcf0dc00 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f702dffb700 1 -- 172.21.15.98:0/2870403578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000ee80 con 0x55a8bcf0dc00 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2870403578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 msgr2=0x55a8bcee90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7040009750 tx=0x7f7040019000 comp rx=0 tx=0).stop 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2870403578 shutdown_connections 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcee90f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcebe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2870403578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcee8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2870403578 >> 172.21.15.98:0/2870403578 conn(0x55a8bcebf160 msgr2=0x55a8bcf02680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2870403578 shutdown_connections 2023-10-06T01:41:20.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.529+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2870403578 wait complete. 2023-10-06T01:41:20.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.530+0000 7f7057c2ab80 1 -- start start 2023-10-06T01:41:20.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.530+0000 7f7057c2ab80 1 Processor -- start 2023-10-06T01:41:20.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.530+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcf6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.530+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcefa560 con 0x55a8bcf0dc00 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcf6ac20 con 0x55a8bcebd840 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcee3ae0 con 0x55a8bcebecc0 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 10 client.0 _ll_register_callbacks cb 0x55a8bce80e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36190/0 (socket says 172.21.15.98:36190) 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 -- 172.21.15.98:0/2400135578 learned_addr learned my addr 172.21.15.98:0/2400135578 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f70477fe700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7047fff700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcf6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 -- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 msgr2=0x55a8bcf5dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf5dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 -- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 msgr2=0x55a8bcf6a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcf6a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 -- 172.21.15.98:0/2400135578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7030003570 con 0x55a8bcf0dc00 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f704cecf700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7040020040 tx=0x7f7040024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff3553b80f76b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f70267fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040023040 con 0x55a8bcf0dc00 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f7040014b80 con 0x55a8bcf0dc00 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f703c058af0 con 0x55a8bcf0dc00 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f704001d440 con 0x55a8bcf0dc00 2023-10-06T01:41:20.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 10 client.5706 fetch_fsmap learned FSMap version 27 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.531+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55a8bcdc3470 con 0x55a8bcf0dc00 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c058af0 con 0x55a8bcf0dc00 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f704001d5a0 con 0x55a8bcf0dc00 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 10 client.5706 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 10 client.5706 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 20 client.5706 populate_metadata read hostname 'smithi098' 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcefea20 0x55a8bcee8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 4 client.5706 mds_command: new command op to 5409 tid=0[{"prefix": "status"}] 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 --> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8bcdb0e00 con 0x55a8bcefea20 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7047fff700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcefea20 0x55a8bcee8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 10 client.5706 ms_handle_connect on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:20.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7047fff700 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcefea20 0x55a8bcee8bb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b376066d456aa96d server_cookie=4837481b29e1f97a in_seq=0 out_seq=0 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 1 -- 172.21.15.98:0/2400135578 <== mds.0 v2:172.21.15.98:6836/1433484145 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55a8bcdb0e00 con 0x55a8bcefea20 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 10 client.5706 handle_command_reply: tid=0 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 client.5706 shutdown 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 msgr2=0x55a8bcefd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7040020040 tx=0x7f7040024000 comp rx=0 tx=0).stop 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 shutdown_connections 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x55a8bcefea20 0x55a8bcee8bb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8bcf0dc00 0x55a8bcefd3b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f70267fc700 0 client.5706 ms_handle_reset on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8bcebd840 0x55a8bcf6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 --2- 172.21.15.98:0/2400135578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a8bcebecc0 0x55a8bcf5dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 >> 172.21.15.98:0/2400135578 conn(0x55a8bcebf160 msgr2=0x55a8bcf033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.532+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 shutdown_connections 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f7057c2ab80 1 -- 172.21.15.98:0/2400135578 wait complete. 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f7057c2ab80 20 client.5706 trim_cache size 0 max 16384 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "c6dd341a-820f-4e37-8897-3250639f93a1", 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2023-10-06T01:41:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5409, 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "recovery_fs", 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 4.5053112999999998, 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 27, 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 44, 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 44, 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 18.99817329 2023-10-06T01:41:20.543 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 msgr2=0x7f70340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 0x7f70340693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70480cebc0 tx=0x7f7038009040 comp rx=0 tx=0).stop 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 msgr2=0x7f70480ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f703c002980 tx=0x7f703c007d30 comp rx=0 tx=0).stop 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 shutdown_connections 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f7034066f30 0x7f70340693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7048155b10 0x7f70481472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70300051c0 0x7f70480ce270 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 --2- 172.21.15.98:0/1044818312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70480ce7b0 0x7f70481555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 >> 172.21.15.98:0/1044818312 conn(0x7f7048070810 msgr2=0x7f7048069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 shutdown_connections 2023-10-06T01:41:20.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.533+0000 7f704e932700 1 -- 172.21.15.98:0/1044818312 wait complete. 2023-10-06T01:41:20.546 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 20 2023-10-06T01:41:20.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 Processor -- start 2023-10-06T01:41:20.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 -- start start 2023-10-06T01:41:20.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12601598e0 0x7f12600ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12600cf610 con 0x7f12600cbcb0 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12600cf8f0 con 0x7f1260159510 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1266494700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12600cfbd0 con 0x7f12601598e0 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1264a31700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f12600cbcb0 0x7f12600ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52418/0 (socket says 172.21.15.98:52418) 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f1264a31700 1 -- 172.21.15.98:0/1024606156 learned_addr learned my addr 172.21.15.98:0/1024606156 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.703+0000 7f125f7fe700 1 --2- 172.21.15.98:0/1024606156 >> v2:172.21.15.184:3300/0 conn(0x7f12601598e0 0x7f12600ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920180594 0 0) 0x7f12600cf610 con 0x7f12600cbcb0 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1248003570 con 0x7f12600cbcb0 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868291921 0 0) 0x7f12600cf8f0 con 0x7f1260159510 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12600cf610 con 0x7f1260159510 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125f7fe700 1 -- 172.21.15.98:0/1024606156 >> v1:172.21.15.184:6789/0 conn(0x7f1260159510 legacy=0x7f12600cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125f7fe700 1 -- 172.21.15.98:0/1024606156 >> v1:172.21.15.98:6789/0 conn(0x7f12600cbcb0 legacy=0x7f12600ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.707 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125f7fe700 1 -- 172.21.15.98:0/1024606156 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600cfeb0 con 0x7f12601598e0 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125f7fe700 1 --2- 172.21.15.98:0/1024606156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600ce820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12540098e0 tx=0x7f12540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca1313a02cbb771 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254011630 con 0x7f12601598e0 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1254011790 con 0x7f12601598e0 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.704+0000 7f125effd700 1 -- 172.21.15.98:0/1024606156 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254011b30 con 0x7f12601598e0 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 -- 172.21.15.98:0/1024606156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 msgr2=0x7f12600ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 --2- 172.21.15.98:0/1024606156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600ce820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12540098e0 tx=0x7f12540091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 -- 172.21.15.98:0/1024606156 shutdown_connections 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 --2- 172.21.15.98:0/1024606156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600ce820 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 -- 172.21.15.98:0/1024606156 >> 172.21.15.98:0/1024606156 conn(0x7f1260089e90 msgr2=0x7f12600997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 -- 172.21.15.98:0/1024606156 shutdown_connections 2023-10-06T01:41:20.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.705+0000 7f1266494700 1 -- 172.21.15.98:0/1024606156 wait complete. 2023-10-06T01:41:20.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.706+0000 7f1266494700 1 Processor -- start 2023-10-06T01:41:20.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.706+0000 7f1266494700 1 -- start start 2023-10-06T01:41:20.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1260159510 0x7f12601481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1260153ac0 con 0x7f12600cbcb0 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600c85d0 con 0x7f12601598e0 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12600c88b0 con 0x7f1260159510 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36194/0 (socket says 172.21.15.98:36194) 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 -- 172.21.15.98:0/2252538481 learned_addr learned my addr 172.21.15.98:0/2252538481 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125f7fe700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1260159510 0x7f12601481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1264a31700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 -- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1260159510 msgr2=0x7f12601481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1260159510 0x7f12601481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 -- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 msgr2=0x7f12600d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 -- 172.21.15.98:0/2252538481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254009420 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f125ffff700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f125000c790 tx=0x7f1250007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250010510 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1250010670 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12600c8b90 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250010a10 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.707+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1260149560 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.708+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12600c90b0 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.709+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f125001b400 con 0x7f12600cbcb0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.709+0000 7f1247fff700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 0x7f124c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.709+0000 7f125f7fe700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 0x7f124c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:20.713 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.709+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f12500847c0 con 0x7f12600cbcb0 2023-10-06T01:41:20.714 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.710+0000 7f125f7fe700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 0x7f124c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12540098e0 tx=0x7f1254017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:20.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.714+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12600c90b0 con 0x7f12600cbcb0 2023-10-06T01:41:20.898 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.897+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f12600c90b0 con 0x7f12600cbcb0 2023-10-06T01:41:20.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.900+0000 7f1247fff700 1 -- 172.21.15.98:0/2252538481 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) v1 ==== 108+0+0 (secure 0 0 0) 0x7f1250051570 con 0x7f12600cbcb0 2023-10-06T01:41:20.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 msgr2=0x7f124c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.904 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 0x7f124c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12540098e0 tx=0x7f1254017000 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 msgr2=0x7f1260153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f125000c790 tx=0x7f1250007e10 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 shutdown_connections 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f124c066f30 0x7f124c0693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12601598e0 0x7f12600d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12600cbcb0 0x7f1260153580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 --2- 172.21.15.98:0/2252538481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1260159510 0x7f12601481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.902+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 >> 172.21.15.98:0/2252538481 conn(0x7f1260089e90 msgr2=0x7f12600c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.903+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 shutdown_connections 2023-10-06T01:41:20.905 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:20.903+0000 7f1266494700 1 -- 172.21.15.98:0/2252538481 wait complete. 2023-10-06T01:41:20.916 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / force,recursive,repair 2023-10-06T01:41:21.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.034+0000 7fe6711b4700 1 Processor -- start 2023-10-06T01:41:21.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.034+0000 7fe6711b4700 1 -- start start 2023-10-06T01:41:21.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6711b4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe66c06d610 0x7fe66c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6711b4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c0cd5b0 con 0x7fe66c157360 2023-10-06T01:41:21.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6711b4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c0cd120 con 0x7fe66c156f90 2023-10-06T01:41:21.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6711b4700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0cc510 con 0x7fe66c06d610 2023-10-06T01:41:21.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe66b7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe66c06d610 0x7fe66c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6709b3700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe66c157360 0x7fe66c0b13c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52420/0 (socket says 172.21.15.98:52420) 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.035+0000 7fe6709b3700 1 -- 172.21.15.98:0/127950793 learned_addr learned my addr 172.21.15.98:0/127950793 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66b7fe700 1 -- 172.21.15.98:0/127950793 >> v1:172.21.15.184:6789/0 conn(0x7fe66c156f90 legacy=0x7fe66c06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66b7fe700 1 -- 172.21.15.98:0/127950793 >> v1:172.21.15.98:6789/0 conn(0x7fe66c157360 legacy=0x7fe66c0b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66affd700 1 -- 172.21.15.98:0/127950793 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842014340 0 0) 0x7fe664001490 con 0x7fe66c157360 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66b7fe700 1 -- 172.21.15.98:0/127950793 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0cc7f0 con 0x7fe66c06d610 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/127950793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c06d610 0x7fe66c0cbe30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6600098e0 tx=0x7fe6600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2bd02e6632a886 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66affd700 1 -- 172.21.15.98:0/127950793 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660009e80 con 0x7fe66c06d610 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66affd700 1 -- 172.21.15.98:0/127950793 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66000b820 con 0x7fe66c06d610 2023-10-06T01:41:21.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe66affd700 1 -- 172.21.15.98:0/127950793 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000bba0 con 0x7fe66c06d610 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe6711b4700 1 -- 172.21.15.98:0/127950793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c06d610 msgr2=0x7fe66c0cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe6711b4700 1 --2- 172.21.15.98:0/127950793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c06d610 0x7fe66c0cbe30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6600098e0 tx=0x7fe6600091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe6711b4700 1 -- 172.21.15.98:0/127950793 shutdown_connections 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe6711b4700 1 --2- 172.21.15.98:0/127950793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c06d610 0x7fe66c0cbe30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.036+0000 7fe6711b4700 1 -- 172.21.15.98:0/127950793 >> 172.21.15.98:0/127950793 conn(0x7fe66c0675f0 msgr2=0x7fe66c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.037+0000 7fe6711b4700 1 -- 172.21.15.98:0/127950793 shutdown_connections 2023-10-06T01:41:21.039 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.037+0000 7fe6711b4700 1 -- 172.21.15.98:0/127950793 wait complete. 2023-10-06T01:41:21.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 Processor -- start 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 -- start start 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 0x7fe66c141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe66c156f90 0x7fe66c0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c1560a0 con 0x7fe66c157360 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c1420e0 con 0x7fe66c06d610 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.038+0000 7fe6711b4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c155490 con 0x7fe66c156f90 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36208/0 (socket says 172.21.15.98:36208) 2023-10-06T01:41:21.042 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 -- 172.21.15.98:0/3417506534 learned_addr learned my addr 172.21.15.98:0/3417506534 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe66c156f90 0x7fe66c0c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 -- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe66c156f90 msgr2=0x7fe66c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe66bfff700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 0x7fe66c141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe66c156f90 0x7fe66c0c3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 -- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 msgr2=0x7fe66c141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 0x7fe66c141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 -- 172.21.15.98:0/3417506534 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660009420 con 0x7fe66c157360 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe66bfff700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 0x7fe66c141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6709b3700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe66400cc90 tx=0x7fe664001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe653fff700 1 -- 172.21.15.98:0/3417506534 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cea0 con 0x7fe66c157360 2023-10-06T01:41:21.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe653fff700 1 -- 172.21.15.98:0/3417506534 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664004bb0 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c155770 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe653fff700 1 -- 172.21.15.98:0/3417506534 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400ab70 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.039+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c155d30 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.041+0000 7fe653fff700 1 -- 172.21.15.98:0/3417506534 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6640070f0 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.041+0000 7fe653fff700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.041+0000 7fe653fff700 1 -- 172.21.15.98:0/3417506534 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fe664084770 con 0x7fe66c157360 2023-10-06T01:41:21.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.041+0000 7fe66bfff700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.042+0000 7fe66bfff700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 0x7fe6580693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe65c002880 tx=0x7fe65c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.043+0000 7fe67b70eb80 1 Processor -- start 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.043+0000 7fe67b70eb80 1 -- start start 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d03b0 0x5651409cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565140925b60 con 0x5651409d1b00 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x565140923360 con 0x5651409d03b0 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56514091f150 con 0x5651409eff40 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36210/0 (socket says 172.21.15.98:36210) 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe6709b3700 1 -- 172.21.15.98:0/1288833657 learned_addr learned my addr 172.21.15.98:0/1288833657 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe66bfff700 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.044+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d03b0 0x5651409cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 -- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 msgr2=0x5651409cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 -- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d03b0 msgr2=0x5651409cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d03b0 0x5651409cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 -- 172.21.15.98:0/1288833657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651407b9360 con 0x5651409d1b00 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6709b3700 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe6640827a0 tx=0x7fe664082730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512553bd8ea5871c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6517fa700 1 -- 172.21.15.98:0/1288833657 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66406b070 con 0x5651409d1b00 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6517fa700 1 -- 172.21.15.98:0/1288833657 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6640552e0 con 0x5651409d1b00 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.045+0000 7fe6517fa700 1 -- 172.21.15.98:0/1288833657 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640584b0 con 0x5651409d1b00 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1288833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 msgr2=0x5651409cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe6640827a0 tx=0x7fe664082730 comp rx=0 tx=0).stop 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1288833657 shutdown_connections 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d1b00 0x5651409cb830 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1288833657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d03b0 0x5651409cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1288833657 >> 172.21.15.98:0/1288833657 conn(0x5651409cff70 msgr2=0x5651409d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1288833657 shutdown_connections 2023-10-06T01:41:21.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1288833657 wait complete. 2023-10-06T01:41:21.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 -- start start 2023-10-06T01:41:21.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.046+0000 7fe67b70eb80 1 Processor -- start 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x565140a181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651408ca980 con 0x5651409eff40 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651408e5f90 con 0x5651409d1b00 2023-10-06T01:41:21.053 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651407a1f20 con 0x5651409d03b0 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe67b70eb80 10 client.0 _ll_register_callbacks cb 0x565140992110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36220/0 (socket says 172.21.15.98:36220) 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 -- 172.21.15.98:0/2356293632 learned_addr learned my addr 172.21.15.98:0/2356293632 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x565140a181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 -- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 msgr2=0x565140a181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x565140a181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 -- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 msgr2=0x565140a16480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a16480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.054 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.047+0000 7fe6709b3700 1 -- 172.21.15.98:0/2356293632 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c155d30 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe664054f10 tx=0x7fe66406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495e9ae11e9b39e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640584b0 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fe664082070 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651409c3a30 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66405bf80 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 10 client.5727 fetch_fsmap learned FSMap version 27 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56514075c0d0 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651409c3a30 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 3659+0+0 (secure 0 0 0) 0x7fe664063020 con 0x5651409eff40 2023-10-06T01:41:21.055 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 10 client.5727 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 10 client.5727 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 20 client.5727 populate_metadata read hostname 'smithi098' 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x565140a2f6d0 0x565140a31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 4 client.5727 mds_command: new command op to 5409 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 --> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651409a0960 con 0x565140a2f6d0 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.048+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x565140a2f6d0 0x565140a31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe64a7fc700 10 client.5727 ms_handle_connect on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x565140a2f6d0 0x565140a31ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f636aacf254c573 server_cookie=be9bb33345b7f818 in_seq=0 out_seq=0 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2356293632 <== mds.0 v2:172.21.15.98:6836/1433484145 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5651409a0960 con 0x565140a2f6d0 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe64a7fc700 10 client.5727 handle_command_reply: tid=0 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 client.5727 shutdown 2023-10-06T01:41:21.056 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 msgr2=0x565140a169c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe664054f10 tx=0x7fe66406b000 comp rx=0 tx=0).stop 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 shutdown_connections 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x565140a2f6d0 0x565140a31ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a169c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe64a7fc700 0 client.5727 ms_handle_reset on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x565140a181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2356293632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a16480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 >> 172.21.15.98:0/2356293632 conn(0x5651409cff70 msgr2=0x5651409fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 shutdown_connections 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2356293632 wait complete. 2023-10-06T01:41:21.057 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.049+0000 7fe67b70eb80 20 client.5727 trim_cache size 0 max 16384 2023-10-06T01:41:21.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.065+0000 7fe67b70eb80 1 Processor -- start 2023-10-06T01:41:21.066 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.065+0000 7fe67b70eb80 1 -- start start 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x5651409d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651409eede0 con 0x5651409d03b0 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651409ede00 con 0x5651409d1b00 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651409cce80 con 0x5651409eff40 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe66bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe66bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36224/0 (socket says 172.21.15.98:36224) 2023-10-06T01:41:21.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe66bfff700 1 -- 172.21.15.98:0/1416018992 learned_addr learned my addr 172.21.15.98:0/1416018992 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.066+0000 7fe6709b3700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x5651409d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 -- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 msgr2=0x5651409c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 -- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 msgr2=0x5651409d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x5651409d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 -- 172.21.15.98:0/1416018992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651408ed870 con 0x5651409d03b0 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x5651409d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe66bfff700 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00d600 tx=0x7fe65c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb07b9c1a46366a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe651ffb700 1 -- 172.21.15.98:0/1416018992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0260b0 con 0x5651409d03b0 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe651ffb700 1 -- 172.21.15.98:0/1416018992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c00ddb0 con 0x5651409d03b0 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.067+0000 7fe651ffb700 1 -- 172.21.15.98:0/1416018992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c007250 con 0x5651409d03b0 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1416018992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 msgr2=0x565140a244d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00d600 tx=0x7fe65c016040 comp rx=0 tx=0).stop 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1416018992 shutdown_connections 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409eff40 0x5651409c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409d03b0 0x565140a244d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/1416018992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x5651409d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1416018992 >> 172.21.15.98:0/1416018992 conn(0x5651409cff70 msgr2=0x5651409fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1416018992 shutdown_connections 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- 172.21.15.98:0/1416018992 wait complete. 2023-10-06T01:41:21.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 -- start start 2023-10-06T01:41:21.072 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.068+0000 7fe67b70eb80 1 Processor -- start 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x5651409f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a7f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565140a70c00 con 0x5651409eff40 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651409f49d0 con 0x5651409d1b00 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651409f4cb0 con 0x5651409d03b0 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe67b70eb80 10 client.0 _ll_register_callbacks cb 0x565140992110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe6709b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36228/0 (socket says 172.21.15.98:36228) 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe6709b3700 1 -- 172.21.15.98:0/2423205434 learned_addr learned my addr 172.21.15.98:0/2423205434 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe66b7fe700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a7f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.069+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x5651409f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 -- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 msgr2=0x5651409f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x5651409f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 -- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 msgr2=0x565140a7f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a7f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 -- 172.21.15.98:0/2423205434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c00db60 con 0x5651409eff40 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe6709b3700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe664054df0 tx=0x7fe664081000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb6eca48620545 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-10-06T01:41:21.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66406b040 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fe664082040 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6640552e0 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66405bf80 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 10 client.5733 fetch_fsmap learned FSMap version 27 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5651408d4560 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6640552e0 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 3659+0+0 (secure 0 0 0) 0x7fe6640554c0 con 0x5651409eff40 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 10 client.5733 fetch_fsmap finished waiting for FSMap version 27 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 10 client.5733 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 20 client.5733 populate_metadata read hostname 'smithi098' 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x5651409cbc40 0x5651409fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 4 client.5733 mds_command: new command op to 5409 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}] 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.070+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 --> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}) v1 -- 0x5651408c1ef0 con 0x5651409cbc40 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.071+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x5651409cbc40 0x5651409fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.071+0000 7fe64a7fc700 10 client.5733 ms_handle_connect on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.071+0000 7fe66bfff700 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x5651409cbc40 0x5651409fac80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3281e8bf13d8f7e server_cookie=6b16fe71b8db5766 in_seq=0 out_seq=0 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe64a7fc700 1 -- 172.21.15.98:0/2423205434 <== mds.0 v2:172.21.15.98:6836/1433484145 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5651408c1ef0 con 0x5651409cbc40 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe64a7fc700 10 client.5733 handle_command_reply: tid=0 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 client.5733 shutdown 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 msgr2=0x565140a7f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe664054df0 tx=0x7fe664081000 comp rx=0 tx=0).stop 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 shutdown_connections 2023-10-06T01:41:21.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:6836/1433484145,v1:172.21.15.98:6837/1433484145] conn(0x5651409cbc40 0x5651409fac80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651409eff40 0x565140a7f840 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651409d03b0 0x5651409f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 --2- 172.21.15.98:0/2423205434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651409d1b00 0x565140a7f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 >> 172.21.15.98:0/2423205434 conn(0x5651409cff70 msgr2=0x565140a16170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe64a7fc700 0 client.5733 ms_handle_reset on v2:172.21.15.98:6836/1433484145 2023-10-06T01:41:21.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 shutdown_connections 2023-10-06T01:41:21.079 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fb8b85ba700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 1 -- 172.21.15.98:0/2423205434 wait complete. 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.072+0000 7fe67b70eb80 20 client.5733 trim_cache size 0 max 16384 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stdout: "return_code": 0, 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_tag": "166e14bd-2d21-4058-b838-1c195b4d2016", 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": "asynchronous" 2023-10-06T01:41:21.080 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:21.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 msgr2=0x7fe6580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 0x7fe6580693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe65c002880 tx=0x7fe65c009040 comp rx=0 tx=0).stop 2023-10-06T01:41:21.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 msgr2=0x7fe66c159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.081 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe66400cc90 tx=0x7fe664001a50 comp rx=0 tx=0).stop 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 shutdown_connections 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe66c157360 0x7fe66c159830 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe66c06d610 0x7fe66c141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 --2- 172.21.15.98:0/3417506534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe66c156f90 0x7fe66c0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 >> 172.21.15.98:0/3417506534 conn(0x7fe66c0675f0 msgr2=0x7fe66c1519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 shutdown_connections 2023-10-06T01:41:21.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fe6711b4700 1 -- 172.21.15.98:0/3417506534 wait complete. 2023-10-06T01:41:21.083 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:21.073+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56522a922b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T01:41:21.083 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:21.074+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56522a99b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/sixmegs v10>,<0x1/subdir v12>]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T01:41:21.083 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:21.077+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 ap=5 snaprealm=0x565229b64c00 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) | dirtyscattered=0 lock=3 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x56522a922580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T01:41:21.086 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:21.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.200+0000 7f545c64c700 1 Processor -- start 2023-10-06T01:41:21.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.200+0000 7f545c64c700 1 -- start start 2023-10-06T01:41:21.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545c64c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.203 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545c64c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f54540b07a0 0x7f54540b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545c64c700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540b0ca0 con 0x7f54540b49e0 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545c64c700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540b0f80 con 0x7f54540b07a0 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545c64c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540b1260 con 0x7f54541595a0 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f545abe9700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f54540b07a0 0x7f54540b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f5459be7700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f54540b49e0 0x7f54540aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52430/0 (socket says 172.21.15.98:52430) 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.201+0000 7f5459be7700 1 -- 172.21.15.98:0/2362790308 learned_addr learned my addr 172.21.15.98:0/2362790308 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545a3e8700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545abe9700 1 -- 172.21.15.98:0/2362790308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 msgr2=0x7f54540aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545abe9700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545abe9700 1 -- 172.21.15.98:0/2362790308 >> v1:172.21.15.98:6789/0 conn(0x7f54540b49e0 legacy=0x7f54540aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545abe9700 1 -- 172.21.15.98:0/2362790308 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540b1540 con 0x7f54540b07a0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f54593e6700 1 -- 172.21.15.98:0/2362790308 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308012620 0 0) 0x7f54540b0ca0 con 0x7f54540b49e0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f545abe9700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b07a0 0x7f54540b1c00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f544c009e60 tx=0x7f544c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b1d25e1a10a5e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f54593e6700 1 -- 172.21.15.98:0/2362790308 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00e070 con 0x7f54540b07a0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f54593e6700 1 -- 172.21.15.98:0/2362790308 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c00bb60 con 0x7f54540b07a0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.202+0000 7f54593e6700 1 -- 172.21.15.98:0/2362790308 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c012430 con 0x7f54540b07a0 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 -- 172.21.15.98:0/2362790308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b07a0 msgr2=0x7f54540b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b07a0 0x7f54540b1c00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f544c009e60 tx=0x7f544c009780 comp rx=0 tx=0).stop 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 -- 172.21.15.98:0/2362790308 shutdown_connections 2023-10-06T01:41:21.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b07a0 0x7f54540b1c00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 --2- 172.21.15.98:0/2362790308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 -- 172.21.15.98:0/2362790308 >> 172.21.15.98:0/2362790308 conn(0x7f5454099830 msgr2=0x7f545406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 -- 172.21.15.98:0/2362790308 shutdown_connections 2023-10-06T01:41:21.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.203+0000 7f545c64c700 1 -- 172.21.15.98:0/2362790308 wait complete. 2023-10-06T01:41:21.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 Processor -- start 2023-10-06T01:41:21.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 -- start start 2023-10-06T01:41:21.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54540b07a0 0x7f545416a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545416aad0 con 0x7f54540b49e0 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545416c070 con 0x7f54540b07a0 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.204+0000 7f545c64c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545416b460 con 0x7f54541595a0 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36236/0 (socket says 172.21.15.98:36236) 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 -- 172.21.15.98:0/1676765999 learned_addr learned my addr 172.21.15.98:0/1676765999 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f545abe9700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 -- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 msgr2=0x7f54540aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f545a3e8700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54540b07a0 0x7f545416a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 -- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54540b07a0 msgr2=0x7f545416a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54540b07a0 0x7f545416a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 -- 172.21.15.98:0/1676765999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c009a20 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f5459be7700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f544400c810 tx=0x7f5444007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444005c00 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444005d60 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545416b740 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444007320 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.205+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545416bd00 con 0x7f54540b49e0 2023-10-06T01:41:21.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.206+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545416c200 con 0x7f54540b49e0 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.207+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f544401b400 con 0x7f54540b49e0 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.207+0000 7f544a7fc700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 0x7f54380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.207+0000 7f545a3e8700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 0x7f54380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.207+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f5444084520 con 0x7f54540b49e0 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.207+0000 7f545a3e8700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 0x7f54380693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5450009390 tx=0x7f5450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.211+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5444051350 con 0x7f54540b49e0 2023-10-06T01:41:21.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.436+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5454064ad0 con 0x7f54540b49e0 2023-10-06T01:41:21.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.437+0000 7f544a7fc700 1 -- 172.21.15.98:0/1676765999 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f5444054970 con 0x7f54540b49e0 2023-10-06T01:41:21.437 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:21.437 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:21.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 msgr2=0x7f54380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 0x7f54380693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5450009390 tx=0x7f5450008040 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 msgr2=0x7f545416ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f544400c810 tx=0x7f5444007db0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 shutdown_connections 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5438066f30 0x7f54380693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54541595a0 0x7f54540aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54540b07a0 0x7f545416a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 --2- 172.21.15.98:0/1676765999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54540b49e0 0x7f545416ac30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 >> 172.21.15.98:0/1676765999 conn(0x7f5454099830 msgr2=0x7f54540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 shutdown_connections 2023-10-06T01:41:21.440 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.439+0000 7f545c64c700 1 -- 172.21.15.98:0/1676765999 wait complete. 2023-10-06T01:41:21.441 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:21.452 INFO:tasks.cephfs.test_recovery_pool:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T01:40:41.845111+0000", "damaged": [], "data_pools": [ 7 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [ 0 ], "flags": 19, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": false }, "fs_name": "cephfs", "in": [ 0 ], "info": {}, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 6, "modified": "2023-10-06T01:40:55.208179+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": {} } }, { "id": 3, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T01:41:04.761696+0000", "damaged": [], "data_pools": [ 7 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "recovery_fs", "in": [ 0 ], "info": { "gid_5409": { "addr": "172.21.15.98:6837/1433484145", "addrs": { "addrvec": [ { "addr": "172.21.15.98:6836", "nonce": 1433484145, "type": "v2" }, { "addr": "172.21.15.98:6837", "nonce": 1433484145, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5409, "incarnation": 24, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 8, "modified": "2023-10-06T01:41:19.034849+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5409 } } } ], "standbys": [ { "addr": "172.21.15.184:6834/2800041409", "addrs": { "addrvec": [ { "addr": "172.21.15.184:6832", "nonce": 2800041409, "type": "v2" }, { "addr": "172.21.15.184:6834", "nonce": 2800041409, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5161, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.184:6835/998571123", "addrs": { "addrvec": [ { "addr": "172.21.15.184:6833", "nonce": 998571123, "type": "v2" }, { "addr": "172.21.15.184:6835", "nonce": 998571123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.98:6835/3383350652", "addrs": { "addrvec": [ { "addr": "172.21.15.98:6834", "nonce": 3383350652, "type": "v2" }, { "addr": "172.21.15.98:6835", "nonce": 3383350652, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5388, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T01:41:21.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T01:41:21.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.567+0000 7fb31f10e700 1 Processor -- start 2023-10-06T01:41:21.568 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.567+0000 7fb31f10e700 1 -- start start 2023-10-06T01:41:21.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31f10e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3181595a0 0x7fb3180aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31f10e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3180b07a0 0x7fb3180b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31f10e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180b0ca0 con 0x7fb3180b49e0 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31f10e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180b0f80 con 0x7fb3180b07a0 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31f10e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180b1260 con 0x7fb3181595a0 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb31d6ab700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3180b07a0 0x7fb3180b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb30ffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb3180b49e0 0x7fb3180aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52446/0 (socket says 172.21.15.98:52446) 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.568+0000 7fb30ffff700 1 -- 172.21.15.98:0/307821601 learned_addr learned my addr 172.21.15.98:0/307821601 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31d6ab700 1 -- 172.21.15.98:0/307821601 >> v2:172.21.15.184:3300/0 conn(0x7fb3181595a0 msgr2=0x7fb3180aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31ceaa700 1 --2- 172.21.15.98:0/307821601 >> v2:172.21.15.184:3300/0 conn(0x7fb3181595a0 0x7fb3180aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb30f7fe700 1 -- 172.21.15.98:0/307821601 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163100981 0 0) 0x7fb3180b0ca0 con 0x7fb3180b49e0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31d6ab700 1 --2- 172.21.15.98:0/307821601 >> v2:172.21.15.184:3300/0 conn(0x7fb3181595a0 0x7fb3180aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31d6ab700 1 -- 172.21.15.98:0/307821601 >> v1:172.21.15.98:6789/0 conn(0x7fb3180b49e0 legacy=0x7fb3180aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31d6ab700 1 -- 172.21.15.98:0/307821601 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180b1540 con 0x7fb3180b07a0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31d6ab700 1 --2- 172.21.15.98:0/307821601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb3180b1c00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3100098e0 tx=0x7fb3100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09723b362f7eeb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb30f7fe700 1 -- 172.21.15.98:0/307821601 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310009e80 con 0x7fb3180b07a0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb30f7fe700 1 -- 172.21.15.98:0/307821601 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31000b820 con 0x7fb3180b07a0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb30f7fe700 1 -- 172.21.15.98:0/307821601 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000bba0 con 0x7fb3180b07a0 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 -- 172.21.15.98:0/307821601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 msgr2=0x7fb3180b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 --2- 172.21.15.98:0/307821601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb3180b1c00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3100098e0 tx=0x7fb3100091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 -- 172.21.15.98:0/307821601 shutdown_connections 2023-10-06T01:41:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 --2- 172.21.15.98:0/307821601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb3180b1c00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 --2- 172.21.15.98:0/307821601 >> v2:172.21.15.184:3300/0 conn(0x7fb3181595a0 0x7fb3180aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 -- 172.21.15.98:0/307821601 >> 172.21.15.98:0/307821601 conn(0x7fb318099830 msgr2=0x7fb31806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 -- 172.21.15.98:0/307821601 shutdown_connections 2023-10-06T01:41:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.569+0000 7fb31f10e700 1 -- 172.21.15.98:0/307821601 wait complete. 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 Processor -- start 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 -- start start 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 0x7fb31815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 0x7fb31809a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181697c0 con 0x7fb3180b07a0 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815ab10 con 0x7fb3181595a0 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.571+0000 7fb31f10e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815adf0 con 0x7fb3180b49e0 2023-10-06T01:41:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36260/0 (socket says 172.21.15.98:36260) 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 -- 172.21.15.98:0/1402067711 learned_addr learned my addr 172.21.15.98:0/1402067711 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31d6ab700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 0x7fb31809a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb30ffff700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 0x7fb31815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 -- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 msgr2=0x7fb31815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 0x7fb31815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 -- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 msgr2=0x7fb31809a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 0x7fb31809a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 -- 172.21.15.98:0/1402067711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310009420 con 0x7fb3180b07a0 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb30ffff700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 0x7fb31815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31d6ab700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 0x7fb31809a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31ceaa700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ceb0 tx=0x7fb314007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400adf0 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31400f450 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31815b0d0 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400f7f0 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.572+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31815d010 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.573+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318064ad0 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.573+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb314003020 con 0x7fb3180b07a0 2023-10-06T01:41:21.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.574+0000 7fb30cff9700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 0x7fb3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.574+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fb314013070 con 0x7fb3180b07a0 2023-10-06T01:41:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.574+0000 7fb30ffff700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 0x7fb3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.574+0000 7fb30ffff700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 0x7fb3040693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb31809dee0 tx=0x7fb308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.578+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb314052140 con 0x7fb3180b07a0 2023-10-06T01:41:21.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.803+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3180b1820 con 0x7fb3180b07a0 2023-10-06T01:41:21.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.803+0000 7fb30cff9700 1 -- 172.21.15.98:0/1402067711 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+156 (secure 0 0 0) 0x7fb314055760 con 0x7fb3180b07a0 2023-10-06T01:41:21.804 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T01:41:21.804 INFO:teuthology.orchestra.run.smithi098.stdout:name: recovery_fs, metadata pool: recovery_fs_metadata, data pools: [cephfs_data ] 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 msgr2=0x7fb3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 0x7fb3040693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb31809dee0 tx=0x7fb308009040 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 msgr2=0x7fb318158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ceb0 tx=0x7fb314007860 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 shutdown_connections 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb304066f30 0x7fb3040693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3181595a0 0x7fb31809a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3180b07a0 0x7fb318158b90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 --2- 172.21.15.98:0/1402067711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3180b49e0 0x7fb31815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 >> 172.21.15.98:0/1402067711 conn(0x7fb318099830 msgr2=0x7fb3180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:21.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 shutdown_connections 2023-10-06T01:41:21.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:21.805+0000 7fb31f10e700 1 -- 172.21.15.98:0/1402067711 wait complete. 2023-10-06T01:41:21.817 INFO:tasks.cephfs.mount:Mounting Ceph FS recovery_fs; just confirmed its presence on cluster 2023-10-06T01:41:21.817 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T01:41:21.817 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:self.cephfs_name = recovery_fs 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T01:41:21.818 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:41:21.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:21.818 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2023-10-06T01:41:21.833 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T01:41:21.833 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T01:41:21.833 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2023-10-06T01:41:21.834 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41856sec preferred_lft 41856sec 2023-10-06T01:41:21.835 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2023-10-06T01:41:21.836 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.836 INFO:teuthology.orchestra.run.smithi098.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T01:41:21.836 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff 2023-10-06T01:41:21.836 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T01:41:21.836 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.837 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c0ec:e9ff:fe71:9101/64 scope link 2023-10-06T01:41:21.837 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.837 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:41:21.837 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 46:42:48:3d:c3:59 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T01:41:21.837 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::4442:48ff:fe3d:c359/64 scope link 2023-10-06T01:41:21.838 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.838 INFO:teuthology.orchestra.run.smithi098.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T01:41:21.838 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether a6:76:1f:1a:8e:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T01:41:21.838 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::a476:1fff:fe1a:8e84/64 scope link 2023-10-06T01:41:21.838 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2023-10-06T01:41:21.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:21.839 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T01:41:21.896 INFO:teuthology.orchestra.run.smithi098.stdout:1 2023-10-06T01:41:21.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:21.898 DEBUG:teuthology.orchestra.run.smithi098:> route 2023-10-06T01:41:21.953 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2023-10-06T01:41:21.954 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T01:41:21.954 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T01:41:21.954 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T01:41:21.954 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T01:41:21.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:21.954 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2023-10-06T01:41:21.954 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2023-10-06T01:41:21.955 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T01:41:21.955 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T01:41:21.955 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T01:41:21.955 DEBUG:teuthology.orchestra.run.smithi098:> ') 2023-10-06T01:41:22.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:22.054 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:41:22.060 INFO:tasks.ceph.mds.a.smithi098.stderr:2023-10-06T01:41:22.059+0000 7fb8bbdc1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:41:22.069 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T01:41:22.069 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T01:41:22.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:41:22.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:22.070 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2023-10-06T01:41:22.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:41:22.134 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T01:41:22.189 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T01:41:22.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T01:41:22.190 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:41:22.244 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T01:41:22.245 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T01:41:22.246 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=recovery_fs 2023-10-06T01:41:22.323 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=recovery_fs 2023-10-06T01:41:22.324 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=recovery_fs". 2023-10-06T01:41:22.324 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:41:22.332 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=recovery_fs 2023-10-06T01:41:22.332 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=recovery_fs". 2023-10-06T01:41:22.332 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:41:22.332 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:41:22.332 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:41:22.354 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=recovery_fs 2023-10-06T01:41:22.354 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=recovery_fs". 2023-10-06T01:41:22.354 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2023-10-06T01:41:22.355 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-06T01:41:22.355 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T01:41:22.355 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790" 2023-10-06T01:41:22.355 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2023-10-06T01:41:22.355 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=recovery_fs,key=0,fsid=c6dd341a-820f-4e37-8897-3250639f93a1" will pass to kernel 2023-10-06T01:41:22.356 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T01:41:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:22.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T01:41:22.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:41:22.422 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir) 2023-10-06T01:41:22.496 INFO:teuthology.orchestra.run.smithi098.stdout:sixmegs 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> import os 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> import stat 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> import json 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> import sys 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> try: 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-06T01:41:22.497 DEBUG:teuthology.orchestra.run.smithi098:> except OSError as e: 2023-10-06T01:41:22.498 DEBUG:teuthology.orchestra.run.smithi098:> sys.exit(e.errno) 2023-10-06T01:41:22.498 DEBUG:teuthology.orchestra.run.smithi098:> 2023-10-06T01:41:22.498 DEBUG:teuthology.orchestra.run.smithi098:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T01:41:22.499 DEBUG:teuthology.orchestra.run.smithi098:> print(json.dumps( 2023-10-06T01:41:22.499 DEBUG:teuthology.orchestra.run.smithi098:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T01:41:22.499 DEBUG:teuthology.orchestra.run.smithi098:> indent=2)) 2023-10-06T01:41:22.499 DEBUG:teuthology.orchestra.run.smithi098:> ' 2023-10-06T01:41:22.619 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:22.619 INFO:teuthology.orchestra.run.smithi098.stdout: "st_mode": 33088, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_ino": 1099511627777, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_dev": 46, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_nlink": 1, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_uid": 0, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_gid": 0, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_size": 6291456, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_atime": 1696556454.0, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_mtime": 1696556454.0, 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout: "st_ctime": 1696556454.0 2023-10-06T01:41:22.620 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:22.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T01:41:22.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T01:41:22.866 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T01:41:22.911 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T01:41:22.912 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T01:41:22.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T01:41:22.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T01:41:22.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:41:22.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:22.954 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T01:41:23.008 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:23.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.161+0000 7f47eac7e700 1 Processor -- start 2023-10-06T01:41:23.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.161+0000 7f47eac7e700 1 -- start start 2023-10-06T01:41:23.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47e41595a0 0x7f47e40aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b49e0 0x7f47e40aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47e40b07a0 0x7f47e40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40b0ca0 con 0x7f47e41595a0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40b0f80 con 0x7f47e40b07a0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47eac7e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40b1260 con 0x7f47e40b49e0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47e8a1a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47e41595a0 0x7f47e40aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47e8a1a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47e41595a0 0x7f47e40aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33532/0 (socket says 172.21.15.98:33532) 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47e8a1a700 1 -- 172.21.15.98:0/2306158492 learned_addr learned my addr 172.21.15.98:0/2306158492 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47dbfff700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b49e0 0x7f47e40aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.162+0000 7f47e921b700 1 --2- 172.21.15.98:0/2306158492 >> v2:172.21.15.184:3300/0 conn(0x7f47e40b07a0 0x7f47e40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 -- 172.21.15.98:0/2306158492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b49e0 msgr2=0x7f47e40aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b49e0 0x7f47e40aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 -- 172.21.15.98:0/2306158492 >> v2:172.21.15.184:3300/0 conn(0x7f47e40b07a0 msgr2=0x7f47e40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/2306158492 >> v2:172.21.15.184:3300/0 conn(0x7f47e40b07a0 0x7f47e40b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 -- 172.21.15.98:0/2306158492 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40b1540 con 0x7f47e41595a0 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e41595a0 0x7f47e40aa570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f47e40d04a0 tx=0x7f47e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ebb6e5f2bb3dc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47db7fe700 1 -- 172.21.15.98:0/2306158492 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0005c10 con 0x7f47e41595a0 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47db7fe700 1 -- 172.21.15.98:0/2306158492 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0005d70 con 0x7f47e41595a0 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47db7fe700 1 -- 172.21.15.98:0/2306158492 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0006260 con 0x7f47e41595a0 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 -- 172.21.15.98:0/2306158492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e41595a0 msgr2=0x7f47e40aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e41595a0 0x7f47e40aa570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f47e40d04a0 tx=0x7f47e0009220 comp rx=0 tx=0).stop 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 -- 172.21.15.98:0/2306158492 shutdown_connections 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 --2- 172.21.15.98:0/2306158492 >> v2:172.21.15.184:3300/0 conn(0x7f47e40b07a0 0x7f47e40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e41595a0 0x7f47e40aa570 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 --2- 172.21.15.98:0/2306158492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b49e0 0x7f47e40aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.163+0000 7f47eac7e700 1 -- 172.21.15.98:0/2306158492 >> 172.21.15.98:0/2306158492 conn(0x7f47e4099830 msgr2=0x7f47e406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:23.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.164+0000 7f47eac7e700 1 -- 172.21.15.98:0/2306158492 shutdown_connections 2023-10-06T01:41:23.167 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.164+0000 7f47eac7e700 1 -- 172.21.15.98:0/2306158492 wait complete. 2023-10-06T01:41:23.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 Processor -- start 2023-10-06T01:41:23.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 -- start start 2023-10-06T01:41:23.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 0x7f47e409b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e41595a0 0x7f47e41565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4159970 con 0x7f47e40b49e0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4159f30 con 0x7f47e41595a0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.165+0000 7f47eac7e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e416d880 con 0x7f47e40b07a0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33540/0 (socket says 172.21.15.98:33540) 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 -- 172.21.15.98:0/1866724468 learned_addr learned my addr 172.21.15.98:0/1866724468 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47e921b700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e41595a0 0x7f47e41565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 0x7f47e409b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 -- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 msgr2=0x7f47e409b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 0x7f47e409b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 -- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e41595a0 msgr2=0x7f47e41565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e41595a0 0x7f47e41565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 -- 172.21.15.98:0/1866724468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0009480 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 0x7f47e409b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47dbfff700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ec80 tx=0x7f47d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400c850 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400c9b0 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e416db60 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400cd70 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.166+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e409dcc0 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.167+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e415a0a0 con 0x7f47e40b49e0 2023-10-06T01:41:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.168+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47d4013680 con 0x7f47e40b49e0 2023-10-06T01:41:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.168+0000 7f47d8ff9700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 0x7f47d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.168+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f47d4084bf0 con 0x7f47e40b49e0 2023-10-06T01:41:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.168+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 0x7f47d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.168+0000 7f47e8a1a700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 0x7f47d00693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009260 tx=0x7f47e0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.172+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47d40519a0 con 0x7f47e40b49e0 2023-10-06T01:41:23.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.397+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47e4064ad0 con 0x7f47e40b49e0 2023-10-06T01:41:23.398 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.397+0000 7f47d8ff9700 1 -- 172.21.15.98:0/1866724468 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f47d4054fc0 con 0x7f47e40b49e0 2023-10-06T01:41:23.398 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:23.398 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:23.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 msgr2=0x7f47d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 0x7f47d00693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009260 tx=0x7f47e0016000 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 msgr2=0x7f47e416ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ec80 tx=0x7f47d40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 shutdown_connections 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f47d0066f30 0x7f47d00693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e41595a0 0x7f47e41565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40b07a0 0x7f47e409b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 --2- 172.21.15.98:0/1866724468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47e40b49e0 0x7f47e416ce00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.399+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 >> 172.21.15.98:0/1866724468 conn(0x7f47e4099830 msgr2=0x7f47e40695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.400+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 shutdown_connections 2023-10-06T01:41:23.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.400+0000 7f47eac7e700 1 -- 172.21.15.98:0/1866724468 wait complete. 2023-10-06T01:41:23.402 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:23.412 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:23.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.525+0000 7f52e653b700 1 Processor -- start 2023-10-06T01:41:23.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 -- start start 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 0x7f52e00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e0068d50 con 0x7f52e00cb630 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e0154060 con 0x7f52e00ca6b0 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e653b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e015a4a0 con 0x7f52e0159b10 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.526+0000 7f52e4ad8700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 0x7f52e00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52dffff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f52e00cb630 0x7f52e00691e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:58936/0 (socket says 172.21.15.98:58936) 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52dffff700 1 -- 172.21.15.98:0/2438975757 learned_addr learned my addr 172.21.15.98:0/2438975757 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:23.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 --2- 172.21.15.98:0/2438975757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 -- 172.21.15.98:0/2438975757 >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 msgr2=0x7f52e00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 --2- 172.21.15.98:0/2438975757 >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 0x7f52e00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 -- 172.21.15.98:0/2438975757 >> v1:172.21.15.184:6789/0 conn(0x7f52e00cb630 legacy=0x7f52e00691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52deffd700 1 -- 172.21.15.98:0/2438975757 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035846066 0 0) 0x7f52e0068d50 con 0x7f52e00cb630 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 -- 172.21.15.98:0/2438975757 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e015a6b0 con 0x7f52e0159b10 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52e4ad8700 1 --2- 172.21.15.98:0/2438975757 >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 0x7f52e00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52df7fe700 1 --2- 172.21.15.98:0/2438975757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0159f60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098e0 tx=0x7f52d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7542b12105ecd710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52deffd700 1 -- 172.21.15.98:0/2438975757 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400b640 con 0x7f52e0159b10 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52deffd700 1 -- 172.21.15.98:0/2438975757 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400b7a0 con 0x7f52e0159b10 2023-10-06T01:41:23.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.527+0000 7f52deffd700 1 -- 172.21.15.98:0/2438975757 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400bb40 con 0x7f52e0159b10 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 -- 172.21.15.98:0/2438975757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 msgr2=0x7f52e0159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 --2- 172.21.15.98:0/2438975757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0159f60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098e0 tx=0x7f52d40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 -- 172.21.15.98:0/2438975757 shutdown_connections 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 --2- 172.21.15.98:0/2438975757 >> v2:172.21.15.184:3300/0 conn(0x7f52e00ca6b0 0x7f52e00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 --2- 172.21.15.98:0/2438975757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0159f60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 -- 172.21.15.98:0/2438975757 >> 172.21.15.98:0/2438975757 conn(0x7f52e0070810 msgr2=0x7f52e0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 -- 172.21.15.98:0/2438975757 shutdown_connections 2023-10-06T01:41:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.528+0000 7f52e653b700 1 -- 172.21.15.98:0/2438975757 wait complete. 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.529+0000 7f52e653b700 1 Processor -- start 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.529+0000 7f52e653b700 1 -- start start 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e00ca6b0 0x7f52e01454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0159800 con 0x7f52e00cb630 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0154640 con 0x7f52e00ca6b0 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e653b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b3480 con 0x7f52e0159b10 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33552/0 (socket says 172.21.15.98:33552) 2023-10-06T01:41:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 -- 172.21.15.98:0/868920881 learned_addr learned my addr 172.21.15.98:0/868920881 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52dffff700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e00ca6b0 0x7f52e01454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e4ad8700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 -- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 msgr2=0x7f52e0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 -- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e00ca6b0 msgr2=0x7f52e01454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e00ca6b0 0x7f52e01454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52df7fe700 1 -- 172.21.15.98:0/868920881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8009a20 con 0x7f52e00cb630 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.530+0000 7f52e4ad8700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52df7fe700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f52e015a5e0 tx=0x7f52d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400b910 con 0x7f52e00cb630 2023-10-06T01:41:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400ba90 con 0x7f52e00cb630 2023-10-06T01:41:23.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4025670 con 0x7f52e00cb630 2023-10-06T01:41:23.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d4009420 con 0x7f52e00cb630 2023-10-06T01:41:23.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.531+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e0159520 con 0x7f52e00cb630 2023-10-06T01:41:23.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.532+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e01547b0 con 0x7f52e00cb630 2023-10-06T01:41:23.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.532+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52d4023070 con 0x7f52e00cb630 2023-10-06T01:41:23.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.532+0000 7f52c7fff700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.533+0000 7f52dffff700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.533+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f52d4085fc0 con 0x7f52e00cb630 2023-10-06T01:41:23.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.533+0000 7f52dffff700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 0x7f52cc0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52d00011e0 tx=0x7f52d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.537+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52d40b75a0 con 0x7f52e00cb630 2023-10-06T01:41:23.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.721+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52e0064ad0 con 0x7f52e00cb630 2023-10-06T01:41:23.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.721+0000 7f52c7fff700 1 -- 172.21.15.98:0/868920881 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f52d4052d70 con 0x7f52e00cb630 2023-10-06T01:41:23.722 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:23.722 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":44,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:06.067115+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:23.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 msgr2=0x7f52cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 0x7f52cc0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52d00011e0 tx=0x7f52d0009040 comp rx=0 tx=0).stop 2023-10-06T01:41:23.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 msgr2=0x7f52e01586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f52e015a5e0 tx=0x7f52d4017000 comp rx=0 tx=0).stop 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 shutdown_connections 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f52cc066f30 0x7f52cc0693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52e0159b10 0x7f52e0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e00ca6b0 0x7f52e01454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 --2- 172.21.15.98:0/868920881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52e00cb630 0x7f52e01586d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.723+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 >> 172.21.15.98:0/868920881 conn(0x7f52e0070810 msgr2=0x7f52e0151790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.724+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 shutdown_connections 2023-10-06T01:41:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.724+0000 7f52e653b700 1 -- 172.21.15.98:0/868920881 wait complete. 2023-10-06T01:41:23.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T01:40:39.889076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:40:40.800463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}}, {'pool': 8, 'pool_name': 'recovery_fs_metadata', 'create_time': '2023-10-06T01:41:03.494206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}}] 2023-10-06T01:41:23.737 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:41:23.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.855+0000 7f862759e700 1 Processor -- start 2023-10-06T01:41:23.856 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.855+0000 7f862759e700 1 -- start start 2023-10-06T01:41:23.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86280b2660 0x7f86280cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862759e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f86280cf1f0 0x7f86280cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862759e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280cc9b0 con 0x7f86280b2a30 2023-10-06T01:41:23.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862759e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86280cbda0 con 0x7f86280cf1f0 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86280cc080 con 0x7f86280b2660 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f86280cf1f0 0x7f86280cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8625d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f86280b2a30 0x7f86280ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52066/0 (socket says 172.21.15.98:52066) 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8625d9b700 1 -- 172.21.15.98:0/3952655736 learned_addr learned my addr 172.21.15.98:0/3952655736 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 -- 172.21.15.98:0/3952655736 >> v2:172.21.15.184:3300/0 conn(0x7f86280b2660 msgr2=0x7f86280cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862659c700 1 --2- 172.21.15.98:0/3952655736 >> v2:172.21.15.184:3300/0 conn(0x7f86280b2660 0x7f86280cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 --2- 172.21.15.98:0/3952655736 >> v2:172.21.15.184:3300/0 conn(0x7f86280b2660 0x7f86280cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 -- 172.21.15.98:0/3952655736 >> v1:172.21.15.98:6789/0 conn(0x7f86280b2a30 legacy=0x7f86280ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f862559a700 1 -- 172.21.15.98:0/3952655736 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380154002 0 0) 0x7f86280cc9b0 con 0x7f86280b2a30 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 -- 172.21.15.98:0/3952655736 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280cc360 con 0x7f86280cf1f0 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.856+0000 7f8626d9d700 1 --2- 172.21.15.98:0/3952655736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f86280cd230 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8628089500 tx=0x7f8620009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56da04a4ab672662 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862559a700 1 -- 172.21.15.98:0/3952655736 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862000e070 con 0x7f86280cf1f0 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862559a700 1 -- 172.21.15.98:0/3952655736 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862000bc30 con 0x7f86280cf1f0 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862559a700 1 -- 172.21.15.98:0/3952655736 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620012430 con 0x7f86280cf1f0 2023-10-06T01:41:23.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 -- 172.21.15.98:0/3952655736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 msgr2=0x7f86280cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 --2- 172.21.15.98:0/3952655736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f86280cd230 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8628089500 tx=0x7f8620009780 comp rx=0 tx=0).stop 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 -- 172.21.15.98:0/3952655736 shutdown_connections 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 --2- 172.21.15.98:0/3952655736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f86280cd230 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 --2- 172.21.15.98:0/3952655736 >> v2:172.21.15.184:3300/0 conn(0x7f86280b2660 0x7f86280cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 -- 172.21.15.98:0/3952655736 >> 172.21.15.98:0/3952655736 conn(0x7f8628099560 msgr2=0x7f862806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 -- 172.21.15.98:0/3952655736 shutdown_connections 2023-10-06T01:41:23.862 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.857+0000 7f862759e700 1 -- 172.21.15.98:0/3952655736 wait complete. 2023-10-06T01:41:23.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.858+0000 7f862759e700 1 Processor -- start 2023-10-06T01:41:23.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.858+0000 7f862759e700 1 -- start start 2023-10-06T01:41:23.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86280b2660 0x7f8628169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280b2a30 0x7f862816a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280b5350 con 0x7f86280cf1f0 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862816a540 con 0x7f86280b2660 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862816baf0 con 0x7f86280b2a30 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33568/0 (socket says 172.21.15.98:33568) 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 -- 172.21.15.98:0/3944156605 learned_addr learned my addr 172.21.15.98:0/3944156605 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f862659c700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86280b2660 0x7f8628169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8625d9b700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280b2a30 0x7f862816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 -- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280b2a30 msgr2=0x7f862816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280b2a30 0x7f862816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 -- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86280b2660 msgr2=0x7f8628169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86280b2660 0x7f8628169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.859+0000 7f8626d9d700 1 -- 172.21.15.98:0/3944156605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620009a20 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f8626d9d700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f862806fd00 tx=0x7f8620019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620012e80 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862001d630 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862816ac40 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862001d9b0 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.860+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862816b1c0 con 0x7f86280cf1f0 2023-10-06T01:41:23.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.861+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862805f270 con 0x7f86280cf1f0 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.861+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8620024070 con 0x7f86280cf1f0 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.862+0000 7f86167fc700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 0x7f86080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.862+0000 7f862659c700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 0x7f86080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.862+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f86200872f0 con 0x7f86280cf1f0 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.862+0000 7f862659c700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 0x7f86080693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f862815dc10 tx=0x7f8618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:23.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:23.866+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86200540a0 con 0x7f86280cf1f0 2023-10-06T01:41:24.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.046+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8628069cb0 con 0x7f86280cf1f0 2023-10-06T01:41:24.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.047+0000 7f86167fc700 1 -- 172.21.15.98:0/3944156605 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f86200576c0 con 0x7f86280cf1f0 2023-10-06T01:41:24.048 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 msgr2=0x7f86080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 0x7f86080693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f862815dc10 tx=0x7f8618008040 comp rx=0 tx=0).stop 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 msgr2=0x7f862816a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f862806fd00 tx=0x7f8620019000 comp rx=0 tx=0).stop 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 shutdown_connections 2023-10-06T01:41:24.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8608066f30 0x7f86080693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86280cf1f0 0x7f862816a700 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86280b2660 0x7f8628169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 --2- 172.21.15.98:0/3944156605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280b2a30 0x7f862816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 >> 172.21.15.98:0/3944156605 conn(0x7f8628099560 msgr2=0x7f8628156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 shutdown_connections 2023-10-06T01:41:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.049+0000 7f862759e700 1 -- 172.21.15.98:0/3944156605 wait complete. 2023-10-06T01:41:24.063 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:41:24.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.178+0000 7fbe59ea1700 1 Processor -- start 2023-10-06T01:41:24.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.178+0000 7fbe59ea1700 1 -- start start 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe59ea1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe540cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe59ea1700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe540cbae0 0x7fbe540cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe59ea1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540cbf00 con 0x7fbe541598e0 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe59ea1700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe540cf610 con 0x7fbe540cbae0 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe59ea1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540cf8f0 con 0x7fbe54159510 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe540cbae0 0x7fbe540cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.181 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe540cbae0 0x7fbe540cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33578/0 (socket says 172.21.15.98:33578) 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 -- 172.21.15.98:0/2922607292 learned_addr learned my addr 172.21.15.98:0/2922607292 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe537fe700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe540cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 -- 172.21.15.98:0/2922607292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 msgr2=0x7fbe540cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe540cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 -- 172.21.15.98:0/2922607292 >> v1:172.21.15.184:6789/0 conn(0x7fbe541598e0 legacy=0x7fbe540ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe53fff700 1 -- 172.21.15.98:0/2922607292 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540cfbd0 con 0x7fbe540cbae0 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.179+0000 7fbe537fe700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe540cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe53fff700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe540cbae0 0x7fbe540cef30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009f20 tx=0x7fbe4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ac9f3321603b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe527fc700 1 -- 172.21.15.98:0/2922607292 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00e070 con 0x7fbe540cbae0 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe527fc700 1 -- 172.21.15.98:0/2922607292 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00ba40 con 0x7fbe540cbae0 2023-10-06T01:41:24.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe527fc700 1 -- 172.21.15.98:0/2922607292 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0092e0 con 0x7fbe540cbae0 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 -- 172.21.15.98:0/2922607292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe540cbae0 msgr2=0x7fbe540cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe540cbae0 0x7fbe540cef30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009f20 tx=0x7fbe4c009840 comp rx=0 tx=0).stop 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 -- 172.21.15.98:0/2922607292 shutdown_connections 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe540cbae0 0x7fbe540cef30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/2922607292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe540cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 -- 172.21.15.98:0/2922607292 >> 172.21.15.98:0/2922607292 conn(0x7fbe54089e90 msgr2=0x7fbe540997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 -- 172.21.15.98:0/2922607292 shutdown_connections 2023-10-06T01:41:24.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.180+0000 7fbe59ea1700 1 -- 172.21.15.98:0/2922607292 wait complete. 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.181+0000 7fbe59ea1700 1 Processor -- start 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 -- start start 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540cbae0 0x7fbe54149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe5416a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5415ac80 con 0x7fbe541598e0 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5416a850 con 0x7fbe540cbae0 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe59ea1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5415a070 con 0x7fbe54159510 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe53fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe53fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33582/0 (socket says 172.21.15.98:33582) 2023-10-06T01:41:24.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe53fff700 1 -- 172.21.15.98:0/546314670 learned_addr learned my addr 172.21.15.98:0/546314670 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe537fe700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540cbae0 0x7fbe54149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.182+0000 7fbe52ffd700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe5416a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 -- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 msgr2=0x7fbe5416a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe5416a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 -- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540cbae0 msgr2=0x7fbe54149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540cbae0 0x7fbe54149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 -- 172.21.15.98:0/546314670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c009ae0 con 0x7fbe541598e0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe53fff700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe54067c60 tx=0x7fbe4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0092e0 con 0x7fbe541598e0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c012d00 con 0x7fbe541598e0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5415a350 con 0x7fbe541598e0 2023-10-06T01:41:24.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c01d650 con 0x7fbe541598e0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.183+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5415a910 con 0x7fbe541598e0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.184+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe54064ad0 con 0x7fbe541598e0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.184+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe4c01da00 con 0x7fbe541598e0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.185+0000 7fbe3b7fe700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 0x7fbe400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.185+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fbe4c086260 con 0x7fbe541598e0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.185+0000 7fbe537fe700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 0x7fbe400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.185+0000 7fbe537fe700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 0x7fbe400693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbe540cf1c0 tx=0x7fbe44006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.189+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe4c053010 con 0x7fbe541598e0 2023-10-06T01:41:24.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.370+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe5415adc0 con 0x7fbe541598e0 2023-10-06T01:41:24.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.370+0000 7fbe3b7fe700 1 -- 172.21.15.98:0/546314670 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe5415adc0 con 0x7fbe541598e0 2023-10-06T01:41:24.371 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:41:24.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 msgr2=0x7fbe400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 0x7fbe400693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbe540cf1c0 tx=0x7fbe44006f90 comp rx=0 tx=0).stop 2023-10-06T01:41:24.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 msgr2=0x7fbe5415b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe54067c60 tx=0x7fbe4c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 shutdown_connections 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fbe40066f30 0x7fbe400693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe541598e0 0x7fbe5415b990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe540cbae0 0x7fbe54149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 --2- 172.21.15.98:0/546314670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe54159510 0x7fbe5416a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 >> 172.21.15.98:0/546314670 conn(0x7fbe54089e90 msgr2=0x7fbe5414ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 shutdown_connections 2023-10-06T01:41:24.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.372+0000 7fbe59ea1700 1 -- 172.21.15.98:0/546314670 wait complete. 2023-10-06T01:41:24.385 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T01:41:24.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 Processor -- start 2023-10-06T01:41:24.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 -- start start 2023-10-06T01:41:24.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c680ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680cf610 con 0x7f2c68159510 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680cf8f0 con 0x7f2c680cbcb0 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.503+0000 7f2c6e016700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cfbd0 con 0x7f2c681598e0 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c677fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c68159510 0x7f2c680cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52068/0 (socket says 172.21.15.98:52068) 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c677fe700 1 -- 172.21.15.98:0/2075483725 learned_addr learned my addr 172.21.15.98:0/2075483725 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821763034 0 0) 0x7f2c680cf610 con 0x7f2c68159510 2023-10-06T01:41:24.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c66ffd700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c680ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50003570 con 0x7f2c68159510 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668024216 0 0) 0x7f2c50003570 con 0x7f2c68159510 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 msgr2=0x7f2c680ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c680ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 >> v1:172.21.15.184:6789/0 conn(0x7f2c680cbcb0 legacy=0x7f2c680ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680cfeb0 con 0x7f2c68159510 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f2c58001470 con 0x7f2c68159510 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 >> v1:172.21.15.98:6789/0 conn(0x7f2c68159510 legacy=0x7f2c680cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.515 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50003a90 0x7f2c50003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c50004d50 0x7f2c500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cfbd0 con 0x7f2c50004d50 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000bb70 con 0x7f2c50003a90 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000bd30 con 0x7f2c681598e0 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c67fff700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c50004d50 0x7f2c500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c66ffd700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.504+0000 7f2c677fe700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50003a90 0x7f2c50003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 -- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 msgr2=0x7f2c50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c681598e0 0x7f2c50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 -- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50003a90 msgr2=0x7f2c50003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50003a90 0x7f2c50003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 -- 172.21.15.98:0/2075483725 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50003570 con 0x7f2c50004d50 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c50004d50 0x7f2c500051c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2c60001f90 tx=0x7f2c60001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff81d69ccc5e389b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60001950 con 0x7f2c50004d50 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6000c8b0 con 0x7f2c50004d50 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c667fc700 1 -- 172.21.15.98:0/2075483725 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000cc50 con 0x7f2c50004d50 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 -- 172.21.15.98:0/2075483725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c50004d50 msgr2=0x7f2c500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 --2- 172.21.15.98:0/2075483725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c50004d50 0x7f2c500051c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2c60001f90 tx=0x7f2c60001640 comp rx=0 tx=0).stop 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c67fff700 1 -- 172.21.15.98:0/2075483725 reap_dead start 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 -- 172.21.15.98:0/2075483725 shutdown_connections 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 -- 172.21.15.98:0/2075483725 >> 172.21.15.98:0/2075483725 conn(0x7f2c68089e90 msgr2=0x7f2c680997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 -- 172.21.15.98:0/2075483725 shutdown_connections 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.505+0000 7f2c6e016700 1 -- 172.21.15.98:0/2075483725 wait complete. 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 Processor -- start 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 -- start start 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6814c720 0x7f2c680b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c681559b0 0x7f2c6815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680b4dc0 con 0x7f2c68159d00 2023-10-06T01:41:24.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68155e20 con 0x7f2c681559b0 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c6e016700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680b4520 con 0x7f2c6814c720 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.507+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33614/0 (socket says 172.21.15.98:33614) 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 -- 172.21.15.98:0/966229096 learned_addr learned my addr 172.21.15.98:0/966229096 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c677fe700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6814c720 0x7f2c680b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c67fff700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c681559b0 0x7f2c6815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 -- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6814c720 msgr2=0x7f2c680b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6814c720 0x7f2c680b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 -- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c681559b0 msgr2=0x7f2c6815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c681559b0 0x7f2c6815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 -- 172.21.15.98:0/966229096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680b4800 con 0x7f2c68159d00 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c66ffd700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011b70 tx=0x7f2c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00fa10 con 0x7f2c68159d00 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00fb70 con 0x7f2c68159d00 2023-10-06T01:41:24.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c014440 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c680b4ae0 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.508+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6814caf0 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.509+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68169d30 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.509+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c5c014870 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.510+0000 7f2c4f7fe700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 0x7f2c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.510+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f2c5c018030 con 0x7f2c68159d00 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.510+0000 7f2c677fe700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 0x7f2c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.510+0000 7f2c677fe700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 0x7f2c540693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c580015a0 tx=0x7f2c58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.514+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c68169d30 con 0x7f2c68159d00 2023-10-06T01:41:24.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.695+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c68169d30 con 0x7f2c68159d00 2023-10-06T01:41:24.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.695+0000 7f2c4f7fe700 1 -- 172.21.15.98:0/966229096 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c68169d30 con 0x7f2c68159d00 2023-10-06T01:41:24.696 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:41:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 msgr2=0x7f2c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 0x7f2c540693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c580015a0 tx=0x7f2c58009040 comp rx=0 tx=0).stop 2023-10-06T01:41:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 msgr2=0x7f2c68155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011b70 tx=0x7f2c5c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 shutdown_connections 2023-10-06T01:41:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c54066f30 0x7f2c540693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c681559b0 0x7f2c6815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6814c720 0x7f2c680b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 --2- 172.21.15.98:0/966229096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c68159d00 0x7f2c68155470 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 >> 172.21.15.98:0/966229096 conn(0x7f2c68089e90 msgr2=0x7f2c68069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 shutdown_connections 2023-10-06T01:41:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.697+0000 7f2c6e016700 1 -- 172.21.15.98:0/966229096 wait complete. 2023-10-06T01:41:24.710 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:41:24.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.827+0000 7f37cb59e700 1 Processor -- start 2023-10-06T01:41:24.828 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.827+0000 7f37cb59e700 1 -- start start 2023-10-06T01:41:24.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cb59e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f37cc0ccab0 0x7f37cc0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cb59e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc0d0c90 con 0x7f37cc1539d0 2023-10-06T01:41:24.830 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cb59e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc0cdc30 con 0x7f37cc0ccab0 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0cb500 con 0x7f37cc0d0830 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37ca59c700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f37cc1539d0 0x7f37cc15a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52078/0 (socket says 172.21.15.98:52078) 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37ca59c700 1 -- 172.21.15.98:0/437257152 learned_addr learned my addr 172.21.15.98:0/437257152 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c3fff700 1 --2- 172.21.15.98:0/437257152 >> v2:172.21.15.98:3300/0 conn(0x7f37cc0ccab0 0x7f37cc0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37cad9d700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c9d9b700 1 -- 172.21.15.98:0/437257152 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950311675 0 0) 0x7f37bc001110 con 0x7f37cc1539d0 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c9d9b700 1 -- 172.21.15.98:0/437257152 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b8003570 con 0x7f37cc1539d0 2023-10-06T01:41:24.831 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c3fff700 1 -- 172.21.15.98:0/437257152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 msgr2=0x7f37cc0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c3fff700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c3fff700 1 -- 172.21.15.98:0/437257152 >> v1:172.21.15.98:6789/0 conn(0x7f37cc1539d0 legacy=0x7f37cc15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.828+0000 7f37c3fff700 1 -- 172.21.15.98:0/437257152 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc0cd020 con 0x7f37cc0ccab0 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37c3fff700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc0ccab0 0x7f37cc0d02f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f37b40098e0 tx=0x7f37b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9d16b8f2099c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37c9d9b700 1 -- 172.21.15.98:0/437257152 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4011630 con 0x7f37cc0ccab0 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37c9d9b700 1 -- 172.21.15.98:0/437257152 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4011790 con 0x7f37cc0ccab0 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37c9d9b700 1 -- 172.21.15.98:0/437257152 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4011b30 con 0x7f37cc0ccab0 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 -- 172.21.15.98:0/437257152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc0ccab0 msgr2=0x7f37cc0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc0ccab0 0x7f37cc0d02f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f37b40098e0 tx=0x7f37b40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.832 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 -- 172.21.15.98:0/437257152 shutdown_connections 2023-10-06T01:41:24.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 --2- 172.21.15.98:0/437257152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc0ccab0 0x7f37cc0d02f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 -- 172.21.15.98:0/437257152 >> 172.21.15.98:0/437257152 conn(0x7f37cc068100 msgr2=0x7f37cc0684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:24.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 -- 172.21.15.98:0/437257152 shutdown_connections 2023-10-06T01:41:24.833 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.829+0000 7f37cb59e700 1 -- 172.21.15.98:0/437257152 wait complete. 2023-10-06T01:41:24.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.830+0000 7f37cb59e700 1 Processor -- start 2023-10-06T01:41:24.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.830+0000 7f37cb59e700 1 -- start start 2023-10-06T01:41:24.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 0x7f37cc1697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc16c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0c8e70 con 0x7f37cc1539d0 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0b18d0 con 0x7f37cc0ccab0 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0b1bb0 con 0x7f37cc0d0830 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33630/0 (socket says 172.21.15.98:33630) 2023-10-06T01:41:24.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cad9d700 1 -- 172.21.15.98:0/850775966 learned_addr learned my addr 172.21.15.98:0/850775966 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:24.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37c3fff700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc16c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37ca59c700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 0x7f37cc1697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.831+0000 7f37cad9d700 1 -- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 msgr2=0x7f37cc16c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cad9d700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc16c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cad9d700 1 -- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 msgr2=0x7f37cc1697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cad9d700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 0x7f37cc1697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cad9d700 1 -- 172.21.15.98:0/850775966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4009420 con 0x7f37cc1539d0 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37c3fff700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc16c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37ca59c700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 0x7f37cc1697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:24.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cad9d700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009df0 tx=0x7f37c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400cb50 con 0x7f37cc1539d0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400f040 con 0x7f37cc1539d0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc0c8820 con 0x7f37cc1539d0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4014480 con 0x7f37cc1539d0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.832+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc0b1510 con 0x7f37cc1539d0 2023-10-06T01:41:24.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.833+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc05f270 con 0x7f37cc1539d0 2023-10-06T01:41:24.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.833+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37c4014830 con 0x7f37cc1539d0 2023-10-06T01:41:24.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.833+0000 7f37c27fc700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:24.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.834+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f37c4085370 con 0x7f37cc1539d0 2023-10-06T01:41:24.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.834+0000 7f37ca59c700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:24.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.834+0000 7f37ca59c700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 0x7f37ac0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f37cc151ff0 tx=0x7f37bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:24.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:24.838+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37c4052120 con 0x7f37cc1539d0 2023-10-06T01:41:25.019 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.019+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37cc0cd300 con 0x7f37cc1539d0 2023-10-06T01:41:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.019+0000 7f37c27fc700 1 -- 172.21.15.98:0/850775966 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37c4055740 con 0x7f37cc1539d0 2023-10-06T01:41:25.020 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:41:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 msgr2=0x7f37ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 0x7f37ac0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f37cc151ff0 tx=0x7f37bc009040 comp rx=0 tx=0).stop 2023-10-06T01:41:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 msgr2=0x7f37cc0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009df0 tx=0x7f37c400b040 comp rx=0 tx=0).stop 2023-10-06T01:41:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 shutdown_connections 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1539d0 0x7f37cc0b0fd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc0ccab0 0x7f37cc1697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 --2- 172.21.15.98:0/850775966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37cc0d0830 0x7f37cc16c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 >> 172.21.15.98:0/850775966 conn(0x7f37cc068100 msgr2=0x7f37cc15c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 shutdown_connections 2023-10-06T01:41:25.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.021+0000 7f37cb59e700 1 -- 172.21.15.98:0/850775966 wait complete. 2023-10-06T01:41:25.033 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2023-10-06T01:41:25.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.150+0000 7f40c9eef700 1 Processor -- start 2023-10-06T01:41:25.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.150+0000 7f40c9eef700 1 -- start start 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c9eef700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f40c41598e0 0x7f40c40ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c9eef700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c40cf610 con 0x7f40c4159510 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c9eef700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c40cf8f0 con 0x7f40c40cbcb0 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c9eef700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40cfbd0 con 0x7f40c41598e0 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c2ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f40c41598e0 0x7f40c40ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c37fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f40c4159510 0x7f40c40cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52084/0 (socket says 172.21.15.98:52084) 2023-10-06T01:41:25.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c37fe700 1 -- 172.21.15.98:0/1265482401 learned_addr learned my addr 172.21.15.98:0/1265482401 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c2ffd700 1 -- 172.21.15.98:0/1265482401 >> v1:172.21.15.184:6789/0 conn(0x7f40c40cbcb0 legacy=0x7f40c40ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c2ffd700 1 -- 172.21.15.98:0/1265482401 >> v1:172.21.15.98:6789/0 conn(0x7f40c4159510 legacy=0x7f40c40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c2ffd700 1 -- 172.21.15.98:0/1265482401 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40cfeb0 con 0x7f40c41598e0 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c27fc700 1 -- 172.21.15.98:0/1265482401 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122740365 0 0) 0x7f40c40cf610 con 0x7f40c4159510 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c2ffd700 1 --2- 172.21.15.98:0/1265482401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c40ce820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098e0 tx=0x7f40b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99eef3a1c5c0bc45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.151+0000 7f40c27fc700 1 -- 172.21.15.98:0/1265482401 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8009e80 con 0x7f40c41598e0 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c27fc700 1 -- 172.21.15.98:0/1265482401 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b800b820 con 0x7f40c41598e0 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c27fc700 1 -- 172.21.15.98:0/1265482401 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b800bba0 con 0x7f40c41598e0 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 -- 172.21.15.98:0/1265482401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 msgr2=0x7f40c40ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 --2- 172.21.15.98:0/1265482401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c40ce820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098e0 tx=0x7f40b80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 -- 172.21.15.98:0/1265482401 shutdown_connections 2023-10-06T01:41:25.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 --2- 172.21.15.98:0/1265482401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c40ce820 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 -- 172.21.15.98:0/1265482401 >> 172.21.15.98:0/1265482401 conn(0x7f40c4089e90 msgr2=0x7f40c40997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 -- 172.21.15.98:0/1265482401 shutdown_connections 2023-10-06T01:41:25.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.152+0000 7f40c9eef700 1 -- 172.21.15.98:0/1265482401 wait complete. 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.153+0000 7f40c9eef700 1 Processor -- start 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.153+0000 7f40c9eef700 1 -- start start 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 0x7f40c40c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4159510 0x7f40c415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c414d0b0 con 0x7f40c41598e0 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c9af0 con 0x7f40c4159510 2023-10-06T01:41:25.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c9eef700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c9dd0 con 0x7f40c40cbcb0 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33660/0 (socket says 172.21.15.98:33660) 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 -- 172.21.15.98:0/3305624464 learned_addr learned my addr 172.21.15.98:0/3305624464 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c2ffd700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4159510 0x7f40c415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c37fe700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 0x7f40c40c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 -- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 msgr2=0x7f40c40c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 0x7f40c40c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 -- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4159510 msgr2=0x7f40c415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4159510 0x7f40c415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c3fff700 1 -- 172.21.15.98:0/3305624464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009420 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.154+0000 7f40c37fe700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 0x7f40c40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40c3fff700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00bb50 tx=0x7f40bc0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00c970 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00cad0 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c414ca60 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00ce90 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.155+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c4149c60 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.156+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c414d220 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.156+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40bc014620 con 0x7f40c41598e0 2023-10-06T01:41:25.160 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.157+0000 7f40ab7fe700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 0x7f40b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.157+0000 7f40c37fe700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 0x7f40b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.157+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f40bc084b10 con 0x7f40c41598e0 2023-10-06T01:41:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.157+0000 7f40c37fe700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 0x7f40b00693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40b40018d0 tx=0x7f40b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.161+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40bc04c5a0 con 0x7f40c41598e0 2023-10-06T01:41:25.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.343+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40c4064ad0 con 0x7f40c41598e0 2023-10-06T01:41:25.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.344+0000 7f40ab7fe700 1 -- 172.21.15.98:0/3305624464 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 109+0+11 (secure 0 0 0) 0x7f40bc0518c0 con 0x7f40c41598e0 2023-10-06T01:41:25.345 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:41:25.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 msgr2=0x7f40b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 0x7f40b00693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40b40018d0 tx=0x7f40b4009040 comp rx=0 tx=0).stop 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 msgr2=0x7f40c4149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00bb50 tx=0x7f40bc0097e0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 shutdown_connections 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40b0066f30 0x7f40b00693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40c41598e0 0x7f40c4149720 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40cbcb0 0x7f40c40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.345+0000 7f40c9eef700 1 --2- 172.21.15.98:0/3305624464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4159510 0x7f40c415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.346+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 >> 172.21.15.98:0/3305624464 conn(0x7f40c4089e90 msgr2=0x7f40c415cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.346+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 shutdown_connections 2023-10-06T01:41:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.346+0000 7f40c9eef700 1 -- 172.21.15.98:0/3305624464 wait complete. 2023-10-06T01:41:25.358 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.472+0000 7f10fd751700 1 Processor -- start 2023-10-06T01:41:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.472+0000 7f10fd751700 1 -- start start 2023-10-06T01:41:25.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10fd751700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10f80caad0 0x7f10f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10fd751700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f10f80675f0 0x7f10f80679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10fd751700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f806ed10 con 0x7f10f815caf0 2023-10-06T01:41:25.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10fd751700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f8067f20 con 0x7f10f80caad0 2023-10-06T01:41:25.481 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10fd751700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80cb8b0 con 0x7f10f80675f0 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10f80caad0 0x7f10f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f6ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f10f815caf0 0x7f10f80caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52092/0 (socket says 172.21.15.98:52092) 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f6ffd700 1 -- 172.21.15.98:0/3394485228 learned_addr learned my addr 172.21.15.98:0/3394485228 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f77fe700 1 --2- 172.21.15.98:0/3394485228 >> v2:172.21.15.184:3300/0 conn(0x7f10f80675f0 0x7f10f80679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 -- 172.21.15.98:0/3394485228 >> v2:172.21.15.184:3300/0 conn(0x7f10f80675f0 msgr2=0x7f10f80679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 --2- 172.21.15.98:0/3394485228 >> v2:172.21.15.184:3300/0 conn(0x7f10f80675f0 0x7f10f80679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.482 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 -- 172.21.15.98:0/3394485228 >> v1:172.21.15.98:6789/0 conn(0x7f10f815caf0 legacy=0x7f10f80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 -- 172.21.15.98:0/3394485228 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80cbb90 con 0x7f10f80caad0 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10effff700 1 --2- 172.21.15.98:0/3394485228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f806f1a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f10f80c8ec0 tx=0x7f10e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5d8895afeeac85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f67fc700 1 -- 172.21.15.98:0/3394485228 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011630 con 0x7f10f80caad0 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f67fc700 1 -- 172.21.15.98:0/3394485228 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0011790 con 0x7f10f80caad0 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.473+0000 7f10f67fc700 1 -- 172.21.15.98:0/3394485228 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011b30 con 0x7f10f80caad0 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 -- 172.21.15.98:0/3394485228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 msgr2=0x7f10f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.483 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 --2- 172.21.15.98:0/3394485228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f806f1a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f10f80c8ec0 tx=0x7f10e00091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 -- 172.21.15.98:0/3394485228 shutdown_connections 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 --2- 172.21.15.98:0/3394485228 >> v2:172.21.15.184:3300/0 conn(0x7f10f80675f0 0x7f10f80679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 --2- 172.21.15.98:0/3394485228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f806f1a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 -- 172.21.15.98:0/3394485228 >> 172.21.15.98:0/3394485228 conn(0x7f10f80cdef0 msgr2=0x7f10f80c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 -- 172.21.15.98:0/3394485228 shutdown_connections 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.474+0000 7f10fd751700 1 -- 172.21.15.98:0/3394485228 wait complete. 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.475+0000 7f10fd751700 1 Processor -- start 2023-10-06T01:41:25.484 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.475+0000 7f10fd751700 1 -- start start 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 0x7f10f80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 0x7f10f816a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b29c0 con 0x7f10f80caad0 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f816ae10 con 0x7f10f80675f0 2023-10-06T01:41:25.485 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10fd751700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b1db0 con 0x7f10f815caf0 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33682/0 (socket says 172.21.15.98:33682) 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 -- 172.21.15.98:0/739546660 learned_addr learned my addr 172.21.15.98:0/739546660 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10f6ffd700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 0x7f10f80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10f77fe700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 0x7f10f816a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 -- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 msgr2=0x7f10f816a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.486 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 0x7f10f816a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 -- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 msgr2=0x7f10f80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 0x7f10f80b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10effff700 1 -- 172.21.15.98:0/739546660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0009420 con 0x7f10f80caad0 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10f77fe700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 0x7f10f816a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.476+0000 7f10f6ffd700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 0x7f10f80b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:25.487 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10effff700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10f806df60 tx=0x7f10e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e001cdc0 con 0x7f10f80caad0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e001a450 con 0x7f10f80caad0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80b2090 con 0x7f10f80caad0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e001a7f0 con 0x7f10f80caad0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.477+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80b2650 con 0x7f10f80caad0 2023-10-06T01:41:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.478+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f80b2b30 con 0x7f10f80caad0 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.478+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10e001abe0 con 0x7f10f80caad0 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.478+0000 7f10eeffd700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 0x7f10d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.479+0000 7f10f6ffd700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 0x7f10d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.479+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f10e00866c0 con 0x7f10f80caad0 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.479+0000 7f10f6ffd700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 0x7f10d80693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10e80058d0 tx=0x7f10e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.483+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10e0053470 con 0x7f10f80caad0 2023-10-06T01:41:25.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.707+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10f8064ad0 con 0x7f10f80caad0 2023-10-06T01:41:25.708 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.708+0000 7f10eeffd700 1 -- 172.21.15.98:0/739546660 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f10e0056a90 con 0x7f10f80caad0 2023-10-06T01:41:25.709 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:25.709 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:25.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 msgr2=0x7f10d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 0x7f10d80693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10e80058d0 tx=0x7f10e8009040 comp rx=0 tx=0).stop 2023-10-06T01:41:25.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 msgr2=0x7f10f815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10f806df60 tx=0x7f10e0016000 comp rx=0 tx=0).stop 2023-10-06T01:41:25.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 shutdown_connections 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f10d8066f30 0x7f10d80693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10f815caf0 0x7f10f816a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f80675f0 0x7f10f80b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 --2- 172.21.15.98:0/739546660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10f80caad0 0x7f10f815af80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 >> 172.21.15.98:0/739546660 conn(0x7f10f80cdef0 msgr2=0x7f10f815b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 shutdown_connections 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.710+0000 7f10fd751700 1 -- 172.21.15.98:0/739546660 wait complete. 2023-10-06T01:41:25.712 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:25.722 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:25.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.836+0000 7f410f95f700 1 Processor -- start 2023-10-06T01:41:25.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.836+0000 7f410f95f700 1 -- start start 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410f95f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 0x7f41080aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410f95f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f41080b07a0 0x7f41080b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410f95f700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080b0ca0 con 0x7f41081595a0 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410f95f700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080b0f80 con 0x7f41080b07a0 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410f95f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080b1260 con 0x7f41080b49e0 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410d6fb700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f41081595a0 0x7f41080aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52094/0 (socket says 172.21.15.98:52094) 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f41080b07a0 0x7f41080b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410d6fb700 1 -- 172.21.15.98:0/2699104700 learned_addr learned my addr 172.21.15.98:0/2699104700 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:25.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410cefa700 1 --2- 172.21.15.98:0/2699104700 >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 0x7f41080aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f4107fff700 1 -- 172.21.15.98:0/2699104700 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460769412 0 0) 0x7f41080b0ca0 con 0x7f41081595a0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f4107fff700 1 -- 172.21.15.98:0/2699104700 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f0003570 con 0x7f41081595a0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 -- 172.21.15.98:0/2699104700 >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 msgr2=0x7f41080aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 --2- 172.21.15.98:0/2699104700 >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 0x7f41080aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 -- 172.21.15.98:0/2699104700 >> v1:172.21.15.98:6789/0 conn(0x7f41081595a0 legacy=0x7f41080aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 -- 172.21.15.98:0/2699104700 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080b1540 con 0x7f41080b07a0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410cefa700 1 --2- 172.21.15.98:0/2699104700 >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 0x7f41080aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f410defc700 1 --2- 172.21.15.98:0/2699104700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41080b07a0 0x7f41080b1c00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f40f80098e0 tx=0x7f40f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a80833b4c4e7312 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f4107fff700 1 -- 172.21.15.98:0/2699104700 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011630 con 0x7f41080b07a0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f4107fff700 1 -- 172.21.15.98:0/2699104700 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8011790 con 0x7f41080b07a0 2023-10-06T01:41:25.840 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.837+0000 7f4107fff700 1 -- 172.21.15.98:0/2699104700 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011b10 con 0x7f41080b07a0 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 -- 172.21.15.98:0/2699104700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41080b07a0 msgr2=0x7f41080b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 --2- 172.21.15.98:0/2699104700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41080b07a0 0x7f41080b1c00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f40f80098e0 tx=0x7f40f80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 -- 172.21.15.98:0/2699104700 shutdown_connections 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 --2- 172.21.15.98:0/2699104700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41080b07a0 0x7f41080b1c00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 --2- 172.21.15.98:0/2699104700 >> v2:172.21.15.184:3300/0 conn(0x7f41080b49e0 0x7f41080aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 -- 172.21.15.98:0/2699104700 >> 172.21.15.98:0/2699104700 conn(0x7f4108099830 msgr2=0x7f410806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 -- 172.21.15.98:0/2699104700 shutdown_connections 2023-10-06T01:41:25.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.838+0000 7f410f95f700 1 -- 172.21.15.98:0/2699104700 wait complete. 2023-10-06T01:41:25.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.839+0000 7f410f95f700 1 Processor -- start 2023-10-06T01:41:25.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.839+0000 7f410f95f700 1 -- start start 2023-10-06T01:41:25.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 0x7f410809cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 0x7f41080cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410809bf10 con 0x7f41081595a0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410809c1f0 con 0x7f41080b49e0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410f95f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410809c4d0 con 0x7f41080b07a0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33708/0 (socket says 172.21.15.98:33708) 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 -- 172.21.15.98:0/2035807128 learned_addr learned my addr 172.21.15.98:0/2035807128 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410d6fb700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 0x7f410809cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410cefa700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 0x7f41080cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 -- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 msgr2=0x7f410809cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 0x7f410809cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 -- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 msgr2=0x7f41080cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:25.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 0x7f41080cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410defc700 1 -- 172.21.15.98:0/2035807128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f8009420 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410d6fb700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 0x7f410809cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.840+0000 7f410cefa700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 0x7f41080cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f410defc700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4108067c60 tx=0x7f40f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f801a430 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f801a590 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f801a950 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410809c7b0 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.841+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410809d4f0 con 0x7f41081595a0 2023-10-06T01:41:25.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.842+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4108064ad0 con 0x7f41081595a0 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.842+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40f801acc0 con 0x7f41081595a0 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.842+0000 7f41057fa700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 0x7f40f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.843+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f40f80868f0 con 0x7f41081595a0 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.843+0000 7f410d6fb700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 0x7f40f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.843+0000 7f410d6fb700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 0x7f40f40693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4100007fd0 tx=0x7f4100009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:25.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:25.847+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40f80536a0 con 0x7f41081595a0 2023-10-06T01:41:26.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.030+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41080b1820 con 0x7f41081595a0 2023-10-06T01:41:26.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.032+0000 7f41057fa700 1 -- 172.21.15.98:0/2035807128 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f40f8056cc0 con 0x7f41081595a0 2023-10-06T01:41:26.032 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:26.033 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":44,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:06.067115+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:26.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 msgr2=0x7f40f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 0x7f40f40693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4100007fd0 tx=0x7f4100009040 comp rx=0 tx=0).stop 2023-10-06T01:41:26.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 msgr2=0x7f410809a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4108067c60 tx=0x7f40f8017000 comp rx=0 tx=0).stop 2023-10-06T01:41:26.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 shutdown_connections 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f40f4066f30 0x7f40f40693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41081595a0 0x7f410809a090 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41080b07a0 0x7f410809cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 --2- 172.21.15.98:0/2035807128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41080b49e0 0x7f41080cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 >> 172.21.15.98:0/2035807128 conn(0x7f4108099830 msgr2=0x7f4108158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 shutdown_connections 2023-10-06T01:41:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.034+0000 7f410f95f700 1 -- 172.21.15.98:0/2035807128 wait complete. 2023-10-06T01:41:26.047 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T01:41:26.047 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:26.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.160+0000 7f08c722d700 1 Processor -- start 2023-10-06T01:41:26.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.160+0000 7f08c722d700 1 -- start start 2023-10-06T01:41:26.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.161+0000 7f08c722d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f08c00cdea0 0x7f08c00c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.161+0000 7f08c722d700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c015b210 con 0x7f08c0157450 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.161+0000 7f08c722d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00ce4f0 con 0x7f08c0157080 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.161+0000 7f08c722d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c015bdb0 con 0x7f08c00cdea0 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c6a2c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f08c00cdea0 0x7f08c00c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5a2a700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f08c0157450 0x7f08c015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52110/0 (socket says 172.21.15.98:52110) 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5a2a700 1 -- 172.21.15.98:0/2271263383 learned_addr learned my addr 172.21.15.98:0/2271263383 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5229700 1 -- 172.21.15.98:0/2271263383 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008124011 0 0) 0x7f08c015b210 con 0x7f08c0157450 2023-10-06T01:41:26.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5229700 1 -- 172.21.15.98:0/2271263383 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ac003570 con 0x7f08c0157450 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c6a2c700 1 -- 172.21.15.98:0/2271263383 >> v1:172.21.15.184:6789/0 conn(0x7f08c0157080 legacy=0x7f08c006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c6a2c700 1 -- 172.21.15.98:0/2271263383 >> v1:172.21.15.98:6789/0 conn(0x7f08c0157450 legacy=0x7f08c015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c6a2c700 1 -- 172.21.15.98:0/2271263383 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00ce7d0 con 0x7f08c00cdea0 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c6a2c700 1 --2- 172.21.15.98:0/2271263383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c00cdea0 0x7f08c00c9fd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f08b80098e0 tx=0x7f08b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeebeb2249876017 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5229700 1 -- 172.21.15.98:0/2271263383 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800b640 con 0x7f08c00cdea0 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5229700 1 -- 172.21.15.98:0/2271263383 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b800b7a0 con 0x7f08c00cdea0 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.162+0000 7f08c5229700 1 -- 172.21.15.98:0/2271263383 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800bb40 con 0x7f08c00cdea0 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 -- 172.21.15.98:0/2271263383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c00cdea0 msgr2=0x7f08c00c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 --2- 172.21.15.98:0/2271263383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c00cdea0 0x7f08c00c9fd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f08b80098e0 tx=0x7f08b80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 -- 172.21.15.98:0/2271263383 shutdown_connections 2023-10-06T01:41:26.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 --2- 172.21.15.98:0/2271263383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c00cdea0 0x7f08c00c9fd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 -- 172.21.15.98:0/2271263383 >> 172.21.15.98:0/2271263383 conn(0x7f08c006b540 msgr2=0x7f08c006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 -- 172.21.15.98:0/2271263383 shutdown_connections 2023-10-06T01:41:26.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.163+0000 7f08c722d700 1 -- 172.21.15.98:0/2271263383 wait complete. 2023-10-06T01:41:26.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.164+0000 7f08c722d700 1 Processor -- start 2023-10-06T01:41:26.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.164+0000 7f08c722d700 1 -- start start 2023-10-06T01:41:26.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 0x7f08c0167990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 0x7f08c0159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0167ed0 con 0x7f08c0157080 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01550f0 con 0x7f08c00cdea0 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01544e0 con 0x7f08c0157450 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33720/0 (socket says 172.21.15.98:33720) 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 -- 172.21.15.98:0/573285723 learned_addr learned my addr 172.21.15.98:0/573285723 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:26.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c6a2c700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 0x7f08c0159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c622b700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 0x7f08c0167990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 -- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 msgr2=0x7f08c0159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 0x7f08c0159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 -- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 msgr2=0x7f08c0167990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 0x7f08c0167990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 -- 172.21.15.98:0/573285723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009420 con 0x7f08c0157080 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c622b700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 0x7f08c0167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c6a2c700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 0x7f08c0159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c5a2a700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c560 tx=0x7f08b0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0005c00 con 0x7f08c0157080 2023-10-06T01:41:26.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0005d60 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c01547c0 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0007300 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.165+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c0154e00 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.166+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c005f270 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.167+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08b00076b0 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.167+0000 7f08b67fc700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 0x7f08a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.167+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f08b0083ee0 con 0x7f08c0157080 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.167+0000 7f08c622b700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 0x7f08a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.167+0000 7f08c622b700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 0x7f08a40693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007fd0 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.171+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08b0050c90 con 0x7f08c0157080 2023-10-06T01:41:26.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.399+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08c00cb420 con 0x7f08c0157080 2023-10-06T01:41:26.400 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.400+0000 7f08b67fc700 1 -- 172.21.15.98:0/573285723 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7f08b0017020 con 0x7f08c0157080 2023-10-06T01:41:26.401 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:26.401 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 msgr2=0x7f08a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 0x7f08a40693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007fd0 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 msgr2=0x7f08c01521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c560 tx=0x7f08b0007dd0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 shutdown_connections 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f08a4066f30 0x7f08a40693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08c0157450 0x7f08c0159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08c00cdea0 0x7f08c0167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 --2- 172.21.15.98:0/573285723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08c0157080 0x7f08c01521f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 >> 172.21.15.98:0/573285723 conn(0x7f08c006b540 msgr2=0x7f08c00adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 shutdown_connections 2023-10-06T01:41:26.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.402+0000 7f08c722d700 1 -- 172.21.15.98:0/573285723 wait complete. 2023-10-06T01:41:26.404 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:26.414 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:26.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.527+0000 7ff66872e700 1 Processor -- start 2023-10-06T01:41:26.528 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.527+0000 7ff66872e700 1 -- start start 2023-10-06T01:41:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff66872e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff6600ce8e0 0x7ff66015af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff66872e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff660157e20 con 0x7ff6600ca3b0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff66872e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600cb610 con 0x7ff6600adba0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff66872e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6600cbbd0 con 0x7ff6600ce8e0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff6664ca700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff6600ca3b0 0x7ff6601582b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52114/0 (socket says 172.21.15.98:52114) 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.528+0000 7ff6664ca700 1 -- 172.21.15.98:0/2872685315 learned_addr learned my addr 172.21.15.98:0/2872685315 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff666ccb700 1 --2- 172.21.15.98:0/2872685315 >> v2:172.21.15.184:3300/0 conn(0x7ff6600ce8e0 0x7ff66015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420157193 0 0) 0x7ff660157e20 con 0x7ff6600ca3b0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff64c003570 con 0x7ff6600ca3b0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64248753 0 0) 0x7ff6600cb610 con 0x7ff6600adba0 2023-10-06T01:41:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff660157e20 con 0x7ff6600adba0 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3281110551 0 0) 0x7ff64c003570 con 0x7ff6600ca3b0 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 >> v2:172.21.15.184:3300/0 conn(0x7ff6600ce8e0 msgr2=0x7ff66015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 --2- 172.21.15.98:0/2872685315 >> v2:172.21.15.184:3300/0 conn(0x7ff6600ce8e0 0x7ff66015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 >> v1:172.21.15.184:6789/0 conn(0x7ff6600adba0 legacy=0x7ff6600ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600cb8f0 con 0x7ff6600ca3b0 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7ff65c0017f0 con 0x7ff6600ca3b0 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 >> v1:172.21.15.98:6789/0 conn(0x7ff6600ca3b0 legacy=0x7ff6601582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c003bf0 0x7ff64c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 0x7ff64c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0051c0 0x7ff64c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600cbbd0 con 0x7ff64c004810 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00c0b0 con 0x7ff64c003bf0 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00c270 con 0x7ff64c0051c0 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 0x7ff64c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff6664ca700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c003bf0 0x7ff64c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff666ccb700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0051c0 0x7ff64c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 -- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0051c0 msgr2=0x7ff64c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0051c0 0x7ff64c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 -- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c003bf0 msgr2=0x7ff64c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c003bf0 0x7ff64c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 -- 172.21.15.98:0/2872685315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c003570 con 0x7ff64c004810 2023-10-06T01:41:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff666ccb700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0051c0 0x7ff64c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff666ccb700 1 -- 172.21.15.98:0/2872685315 reap_dead start 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.529+0000 7ff665cc9700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 0x7ff64c004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff6500019e0 tx=0x7ff650007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f9f0d4b4bb78a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000c5d0 con 0x7ff64c004810 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000c730 con 0x7ff64c004810 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff6654c8700 1 -- 172.21.15.98:0/2872685315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000caf0 con 0x7ff64c004810 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 -- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 msgr2=0x7ff64c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 0x7ff64c004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff6500019e0 tx=0x7ff650007d30 comp rx=0 tx=0).stop 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 -- 172.21.15.98:0/2872685315 shutdown_connections 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 --2- 172.21.15.98:0/2872685315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff64c004810 0x7ff64c004c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 -- 172.21.15.98:0/2872685315 >> 172.21.15.98:0/2872685315 conn(0x7ff660070810 msgr2=0x7ff660070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 -- 172.21.15.98:0/2872685315 shutdown_connections 2023-10-06T01:41:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.530+0000 7ff66872e700 1 -- 172.21.15.98:0/2872685315 wait complete. 2023-10-06T01:41:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.531+0000 7ff66872e700 1 Processor -- start 2023-10-06T01:41:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.531+0000 7ff66872e700 1 -- start start 2023-10-06T01:41:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004810 0x7ff6600b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6600b3610 0x7ff6600b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600ca0e0 con 0x7ff6600c9730 2023-10-06T01:41:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff660158f50 con 0x7ff64c004810 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff66872e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600b3fc0 con 0x7ff6600b3610 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33726/0 (socket says 172.21.15.98:33726) 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 -- 172.21.15.98:0/3112696589 learned_addr learned my addr 172.21.15.98:0/3112696589 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff666ccb700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6600b3610 0x7ff6600b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff6664ca700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004810 0x7ff6600b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 -- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6600b3610 msgr2=0x7ff6600b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6600b3610 0x7ff6600b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 -- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004810 msgr2=0x7ff6600b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004810 0x7ff6600b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 -- 172.21.15.98:0/3112696589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660158620 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.532+0000 7ff665cc9700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00c080 tx=0x7ff650015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000c5d0 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000cea0 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff660158900 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650006820 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6600b1a30 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.533+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660159100 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.534+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff650019070 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.534+0000 7ff6567fc700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 0x7ff6440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.534+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7ff650086230 con 0x7ff6600c9730 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.534+0000 7ff6664ca700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 0x7ff6440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.535+0000 7ff6664ca700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 0x7ff6440693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff65c004ee0 tx=0x7ff65c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.539+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff660159100 con 0x7ff6600c9730 2023-10-06T01:41:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.763+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6601590c0 con 0x7ff6600c9730 2023-10-06T01:41:26.764 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.763+0000 7ff6567fc700 1 -- 172.21.15.98:0/3112696589 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5668 (secure 0 0 0) 0x7ff650052fe0 con 0x7ff6600c9730 2023-10-06T01:41:26.764 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:26.764 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:40:41.845111+0000","modified":"2023-10-06T01:40:55.208179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 msgr2=0x7ff6440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 0x7ff6440693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff65c004ee0 tx=0x7ff65c006040 comp rx=0 tx=0).stop 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 msgr2=0x7ff6600c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00c080 tx=0x7ff650015000 comp rx=0 tx=0).stop 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 shutdown_connections 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7ff644066f30 0x7ff6440693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6600b3610 0x7ff6600b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004810 0x7ff6600b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.767 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 --2- 172.21.15.98:0/3112696589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6600c9730 0x7ff6600c9ba0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 >> 172.21.15.98:0/3112696589 conn(0x7ff660070810 msgr2=0x7ff6600cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.765+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 shutdown_connections 2023-10-06T01:41:26.768 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.766+0000 7ff66872e700 1 -- 172.21.15.98:0/3112696589 wait complete. 2023-10-06T01:41:26.768 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 27 2023-10-06T01:41:26.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:26.892 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.892+0000 7f5fec904700 1 Processor -- start 2023-10-06T01:41:26.893 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.892+0000 7f5fec904700 1 -- start start 2023-10-06T01:41:26.894 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fec904700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.894 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fec904700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fec904700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe815b210 con 0x7f5fe8157450 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fec904700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80ce4f0 con 0x7f5fe815b6a0 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fec904700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80ce7d0 con 0x7f5fe8157080 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe6ffd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f5fe8157450 0x7f5fe806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:58992/0 (socket says 172.21.15.98:58992) 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe6ffd700 1 -- 172.21.15.98:0/3173679940 learned_addr learned my addr 172.21.15.98:0/3173679940 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:59502/0 (socket says 172.21.15.98:59502) 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:51976/0 (socket says 172.21.15.98:51976) 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 -- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 msgr2=0x7f5fe80cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:26.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 -- 172.21.15.98:0/3173679940 >> v1:172.21.15.184:6789/0 conn(0x7f5fe8157450 legacy=0x7f5fe806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe7fff700 1 -- 172.21.15.98:0/3173679940 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe815aea0 con 0x7f5fe815b6a0 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.893+0000 7f5fe77fe700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fe7fff700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00098e0 tx=0x7f5fe00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b0c349f70af17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fe67fc700 1 -- 172.21.15.98:0/3173679940 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000b640 con 0x7f5fe815b6a0 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fe67fc700 1 -- 172.21.15.98:0/3173679940 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000b7a0 con 0x7f5fe815b6a0 2023-10-06T01:41:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fe67fc700 1 -- 172.21.15.98:0/3173679940 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000bb40 con 0x7f5fe815b6a0 2023-10-06T01:41:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 -- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe815b6a0 msgr2=0x7f5fe80c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00098e0 tx=0x7f5fe00091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 -- 172.21.15.98:0/3173679940 shutdown_connections 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe815b6a0 0x7f5fe80c9fd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 --2- 172.21.15.98:0/3173679940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe8157080 0x7f5fe80cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 -- 172.21.15.98:0/3173679940 >> 172.21.15.98:0/3173679940 conn(0x7f5fe806b540 msgr2=0x7f5fe806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 -- 172.21.15.98:0/3173679940 shutdown_connections 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.894+0000 7f5fec904700 1 -- 172.21.15.98:0/3173679940 wait complete. 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 Processor -- start 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 -- start start 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 0x7f5fe8141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe815b6a0 0x7f5fe816a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8142140 con 0x7f5fe8157450 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8159eb0 con 0x7f5fe8157080 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fec904700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c3410 con 0x7f5fe815b6a0 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33732/0 (socket says 172.21.15.98:33732) 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.896+0000 7f5fe6ffd700 1 -- 172.21.15.98:0/587409553 learned_addr learned my addr 172.21.15.98:0/587409553 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe7fff700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe815b6a0 0x7f5fe816a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe77fe700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 0x7f5fe8141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 -- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe815b6a0 msgr2=0x7f5fe816a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe815b6a0 0x7f5fe816a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 -- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 msgr2=0x7f5fe8141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 0x7f5fe8141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 -- 172.21.15.98:0/587409553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009a20 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe77fe700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 0x7f5fe8141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fe6ffd700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c7f0 tx=0x7f5fdc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc005b30 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc005cb0 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe0009420 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc004a80 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.897+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81503d0 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.898+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe815a060 con 0x7f5fe8157450 2023-10-06T01:41:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.898+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fdc0070d0 con 0x7f5fe8157450 2023-10-06T01:41:26.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.899+0000 7f5fcf7fe700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 0x7f5fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:26.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.899+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f5fdc083dd0 con 0x7f5fe8157450 2023-10-06T01:41:26.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.899+0000 7f5fe77fe700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 0x7f5fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:26.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.899+0000 7f5fe77fe700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 0x7f5fd40693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80099f0 tx=0x7f5fd800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:26.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:26.903+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe815a060 con 0x7f5fe8157450 2023-10-06T01:41:27.091 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.090+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fe815a020 con 0x7f5fe8157450 2023-10-06T01:41:27.092 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.091+0000 7f5fcf7fe700 1 -- 172.21.15.98:0/587409553 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f5fdc085090 con 0x7f5fe8157450 2023-10-06T01:41:27.092 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:27.092 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":44,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:06.067115+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T01:40:39.889076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:27.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 msgr2=0x7f5fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 0x7f5fd40693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80099f0 tx=0x7f5fd800b040 comp rx=0 tx=0).stop 2023-10-06T01:41:27.094 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 msgr2=0x7f5fe80c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c7f0 tx=0x7f5fdc007f70 comp rx=0 tx=0).stop 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 shutdown_connections 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5fd4066f30 0x7f5fd40693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe815b6a0 0x7f5fe816a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe8157080 0x7f5fe8141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 --2- 172.21.15.98:0/587409553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe8157450 0x7f5fe80c2ed0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 >> 172.21.15.98:0/587409553 conn(0x7f5fe806b540 msgr2=0x7f5fe80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 shutdown_connections 2023-10-06T01:41:27.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.093+0000 7f5fec904700 1 -- 172.21.15.98:0/587409553 wait complete. 2023-10-06T01:41:27.106 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T01:41:27.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 Processor -- start 2023-10-06T01:41:27.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 -- start start 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01a81595a0 0x7f01a80aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01a80b49e0 0x7f01a80aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80b0ca0 con 0x7f01a80b07a0 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80b0f80 con 0x7f01a80b49e0 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.220+0000 7f01adc87700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80b1260 con 0x7f01a81595a0 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01a80b49e0 0x7f01a80aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01a80b49e0 0x7f01a80aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33736/0 (socket says 172.21.15.98:33736) 2023-10-06T01:41:27.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 -- 172.21.15.98:0/617201330 learned_addr learned my addr 172.21.15.98:0/617201330 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a77fe700 1 --2- 172.21.15.98:0/617201330 >> v2:172.21.15.184:3300/0 conn(0x7f01a81595a0 0x7f01a80aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 -- 172.21.15.98:0/617201330 >> v2:172.21.15.184:3300/0 conn(0x7f01a81595a0 msgr2=0x7f01a80aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 --2- 172.21.15.98:0/617201330 >> v2:172.21.15.184:3300/0 conn(0x7f01a81595a0 0x7f01a80aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 -- 172.21.15.98:0/617201330 >> v1:172.21.15.184:6789/0 conn(0x7f01a80b07a0 legacy=0x7f01a80b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 -- 172.21.15.98:0/617201330 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80b1540 con 0x7f01a80b49e0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a6ffd700 1 --2- 172.21.15.98:0/617201330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a80b49e0 0x7f01a80aaab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f019c0099b0 tx=0x7f019c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483e95a368ab3c77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a67fc700 1 -- 172.21.15.98:0/617201330 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c009ea0 con 0x7f01a80b49e0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a67fc700 1 -- 172.21.15.98:0/617201330 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00b820 con 0x7f01a80b49e0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.221+0000 7f01a67fc700 1 -- 172.21.15.98:0/617201330 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00bbc0 con 0x7f01a80b49e0 2023-10-06T01:41:27.224 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 -- 172.21.15.98:0/617201330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a80b49e0 msgr2=0x7f01a80aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 --2- 172.21.15.98:0/617201330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a80b49e0 0x7f01a80aaab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f019c0099b0 tx=0x7f019c009250 comp rx=0 tx=0).stop 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 -- 172.21.15.98:0/617201330 shutdown_connections 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 --2- 172.21.15.98:0/617201330 >> v2:172.21.15.184:3300/0 conn(0x7f01a81595a0 0x7f01a80aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 --2- 172.21.15.98:0/617201330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a80b49e0 0x7f01a80aaab0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 -- 172.21.15.98:0/617201330 >> 172.21.15.98:0/617201330 conn(0x7f01a8099830 msgr2=0x7f01a806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 -- 172.21.15.98:0/617201330 shutdown_connections 2023-10-06T01:41:27.225 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.222+0000 7f01adc87700 1 -- 172.21.15.98:0/617201330 wait complete. 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.223+0000 7f01adc87700 1 Processor -- start 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.223+0000 7f01adc87700 1 -- start start 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 0x7f01a814a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 0x7f01a81599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815b170 con 0x7f01a81595a0 2023-10-06T01:41:27.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8157020 con 0x7f01a80b49e0 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815a560 con 0x7f01a80b07a0 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33740/0 (socket says 172.21.15.98:33740) 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 -- 172.21.15.98:0/3595566952 learned_addr learned my addr 172.21.15.98:0/3595566952 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a77fe700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 0x7f01a814a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a6ffd700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 0x7f01a81599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 -- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 msgr2=0x7f01a814a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 0x7f01a814a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 -- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 msgr2=0x7f01a81599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:27.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 0x7f01a81599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 -- 172.21.15.98:0/3595566952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0094f0 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a77fe700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 0x7f01a814a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a6ffd700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 0x7f01a81599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01a7fff700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c8e0 tx=0x7f01a0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000ce80 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000a7e0 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.224+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a815a840 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.225+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000ab80 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.225+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a815ae00 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.226+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a815b320 con 0x7f01a81595a0 2023-10-06T01:41:27.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.226+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01a0012420 con 0x7f01a81595a0 2023-10-06T01:41:27.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.226+0000 7f018f7fe700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:27.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.226+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f01a0084d00 con 0x7f01a81595a0 2023-10-06T01:41:27.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.226+0000 7f01a77fe700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:27.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.227+0000 7f01a77fe700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 0x7f01940693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01a816cec0 tx=0x7f0198008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:27.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.230+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01a815b320 con 0x7f01a81595a0 2023-10-06T01:41:27.456 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:27.455+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f01a815b320 con 0x7f01a81595a0 2023-10-06T01:41:28.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.069+0000 7f018f7fe700 1 -- 172.21.15.98:0/3595566952 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f01a815b320 con 0x7f01a81595a0 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 msgr2=0x7f01940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 0x7f01940693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01a816cec0 tx=0x7f0198008040 comp rx=0 tx=0).stop 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 msgr2=0x7f01a8159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c8e0 tx=0x7f01a0001710 comp rx=0 tx=0).stop 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 shutdown_connections 2023-10-06T01:41:28.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0194066f30 0x7f01940693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a81595a0 0x7f01a8159f30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a80b07a0 0x7f01a814a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 --2- 172.21.15.98:0/3595566952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a80b49e0 0x7f01a81599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 >> 172.21.15.98:0/3595566952 conn(0x7f01a8099830 msgr2=0x7f01a8158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 shutdown_connections 2023-10-06T01:41:28.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.072+0000 7f01adc87700 1 -- 172.21.15.98:0/3595566952 wait complete. 2023-10-06T01:41:28.075 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T01:41:28.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T01:41:28.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.200+0000 7fa6a977d700 1 Processor -- start 2023-10-06T01:41:28.201 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.200+0000 7fa6a977d700 1 -- start start 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a977d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 0x7fa6a4157d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a977d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6a4158250 0x7fa6a4158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a977d700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40afbf0 con 0x7fa6a415b980 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a977d700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40afed0 con 0x7fa6a4158250 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a977d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40b01b0 con 0x7fa6a406bff0 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a2ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa6a415b980 0x7fa6a406b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52120/0 (socket says 172.21.15.98:52120) 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a2ffd700 1 -- 172.21.15.98:0/163165039 learned_addr learned my addr 172.21.15.98:0/163165039 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.201+0000 7fa6a37fe700 1 --2- 172.21.15.98:0/163165039 >> v2:172.21.15.98:3300/0 conn(0x7fa6a4158250 0x7fa6a4158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa69a7fc700 1 --2- 172.21.15.98:0/163165039 >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 0x7fa6a4157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.204 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a27fc700 1 -- 172.21.15.98:0/163165039 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254667037 0 0) 0x7fa6a40afbf0 con 0x7fa6a415b980 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a27fc700 1 -- 172.21.15.98:0/163165039 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa690003570 con 0x7fa6a415b980 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a37fe700 1 -- 172.21.15.98:0/163165039 >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 msgr2=0x7fa6a4157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a37fe700 1 --2- 172.21.15.98:0/163165039 >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 0x7fa6a4157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a37fe700 1 -- 172.21.15.98:0/163165039 >> v1:172.21.15.98:6789/0 conn(0x7fa6a415b980 legacy=0x7fa6a406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a37fe700 1 -- 172.21.15.98:0/163165039 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40b0490 con 0x7fa6a4158250 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa69a7fc700 1 --2- 172.21.15.98:0/163165039 >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 0x7fa6a4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a37fe700 1 --2- 172.21.15.98:0/163165039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a4158250 0x7fa6a4158aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40c1d30 tx=0x7fa69c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d277d183a6dc42e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a27fc700 1 -- 172.21.15.98:0/163165039 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00e070 con 0x7fa6a4158250 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a27fc700 1 -- 172.21.15.98:0/163165039 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c00bb60 con 0x7fa6a4158250 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a27fc700 1 -- 172.21.15.98:0/163165039 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c012430 con 0x7fa6a4158250 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 -- 172.21.15.98:0/163165039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a4158250 msgr2=0x7fa6a4158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.205 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 --2- 172.21.15.98:0/163165039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a4158250 0x7fa6a4158aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40c1d30 tx=0x7fa69c009780 comp rx=0 tx=0).stop 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 -- 172.21.15.98:0/163165039 shutdown_connections 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 --2- 172.21.15.98:0/163165039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a4158250 0x7fa6a4158aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 --2- 172.21.15.98:0/163165039 >> v2:172.21.15.184:3300/0 conn(0x7fa6a406bff0 0x7fa6a4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 -- 172.21.15.98:0/163165039 >> 172.21.15.98:0/163165039 conn(0x7fa6a40ce330 msgr2=0x7fa6a40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.202+0000 7fa6a977d700 1 -- 172.21.15.98:0/163165039 shutdown_connections 2023-10-06T01:41:28.206 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.203+0000 7fa6a977d700 1 -- 172.21.15.98:0/163165039 wait complete. 2023-10-06T01:41:28.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 Processor -- start 2023-10-06T01:41:28.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 -- start start 2023-10-06T01:41:28.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6a4158250 0x7fa6a40c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6a415b980 0x7fa6a40c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40c5900 con 0x7fa6a406bff0 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40c2ce0 con 0x7fa6a4158250 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.204+0000 7fa6a977d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40c37b0 con 0x7fa6a415b980 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33760/0 (socket says 172.21.15.98:33760) 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 -- 172.21.15.98:0/309892122 learned_addr learned my addr 172.21.15.98:0/309892122 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a37fe700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6a415b980 0x7fa6a40c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa69a7fc700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6a4158250 0x7fa6a40c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 -- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6a415b980 msgr2=0x7fa6a40c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6a415b980 0x7fa6a40c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 -- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6a4158250 msgr2=0x7fa6a40c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6a4158250 0x7fa6a40c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 -- 172.21.15.98:0/309892122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009a20 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a2ffd700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c590 tx=0x7fa694007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694005b30 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694005cb0 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a40c5bc0 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694007320 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.205+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a40c31f0 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.206+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a40af4f0 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.207+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa69401b3b0 con 0x7fa6a406bff0 2023-10-06T01:41:28.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.207+0000 7fa69b7fe700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:28.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.207+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fa694083e70 con 0x7fa6a406bff0 2023-10-06T01:41:28.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.207+0000 7fa69a7fc700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:28.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.207+0000 7fa69a7fc700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 0x7fa6840693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40cc250 tx=0x7fa68c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:28.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.211+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa694050c20 con 0x7fa6a406bff0 2023-10-06T01:41:28.437 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:28.436+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa6a40635e0 con 0x7fa6a406bff0 2023-10-06T01:41:29.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.074+0000 7fa69b7fe700 1 -- 172.21.15.98:0/309892122 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa6940170a0 con 0x7fa6a406bff0 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 msgr2=0x7fa6840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 0x7fa6840693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40cc250 tx=0x7fa68c008040 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 msgr2=0x7fa6a40c3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c590 tx=0x7fa694007e00 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 shutdown_connections 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6a415b980 0x7fa6a40c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6a406bff0 0x7fa6a40c3d30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 --2- 172.21.15.98:0/309892122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6a4158250 0x7fa6a40c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 >> 172.21.15.98:0/309892122 conn(0x7fa6a40ce330 msgr2=0x7fa6a40c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:29.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 shutdown_connections 2023-10-06T01:41:29.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.077+0000 7fa6a977d700 1 -- 172.21.15.98:0/309892122 wait complete. 2023-10-06T01:41:29.090 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T01:41:29.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.208+0000 7f63d7fff700 1 Processor -- start 2023-10-06T01:41:29.209 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.208+0000 7f63d7fff700 1 -- start start 2023-10-06T01:41:29.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63d80d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63d80d0830 0x7f63d80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d7fff700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d80cdc30 con 0x7f63d81539d0 2023-10-06T01:41:29.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d7fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80d0cb0 con 0x7f63d80d0830 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80cb500 con 0x7f63d80ccab0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d6ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f63d81539d0 0x7f63d815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52134/0 (socket says 172.21.15.98:52134) 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d6ffd700 1 -- 172.21.15.98:0/1212116031 learned_addr learned my addr 172.21.15.98:0/1212116031 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186067059 0 0) 0x7f63d80cdc30 con 0x7f63d81539d0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c4003570 con 0x7f63d81539d0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d77fe700 1 --2- 172.21.15.98:0/1212116031 >> v2:172.21.15.184:3300/0 conn(0x7f63d80d0830 0x7f63d80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63d80d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1011603799 0 0) 0x7f63c4003570 con 0x7f63d81539d0 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 msgr2=0x7f63d80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63d80d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 >> v2:172.21.15.184:3300/0 conn(0x7f63d80d0830 msgr2=0x7f63d80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 --2- 172.21.15.98:0/1212116031 >> v2:172.21.15.184:3300/0 conn(0x7f63d80d0830 0x7f63d80ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80cd020 con 0x7f63d81539d0 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f63c8001470 con 0x7f63d81539d0 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 >> v1:172.21.15.98:6789/0 conn(0x7f63d81539d0 legacy=0x7f63d815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4003a60 0x7f63c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4004810 0x7f63c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80d0cb0 con 0x7f63c4004810 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80cb500 con 0x7f63c4003a60 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.209+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c400ba60 con 0x7f63d80ccab0 2023-10-06T01:41:29.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4004810 0x7f63c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63d6ffd700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4003a60 0x7f63c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63d77fe700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 -- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 msgr2=0x7f63c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80ccab0 0x7f63c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 -- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4003a60 msgr2=0x7f63c4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4003a60 0x7f63c4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 -- 172.21.15.98:0/1212116031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c4003570 con 0x7f63c4004810 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4004810 0x7f63c4004c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009c30 tx=0x7f63c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7beaf07b0db81c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000e710 con 0x7f63c4004810 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c000e870 con 0x7f63c4004810 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.210+0000 7f63d67fc700 1 -- 172.21.15.98:0/1212116031 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000ec30 con 0x7f63c4004810 2023-10-06T01:41:29.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 -- 172.21.15.98:0/1212116031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4004810 msgr2=0x7f63c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 --2- 172.21.15.98:0/1212116031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4004810 0x7f63c4004c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009c30 tx=0x7f63c00091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d77fe700 1 -- 172.21.15.98:0/1212116031 reap_dead start 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 -- 172.21.15.98:0/1212116031 shutdown_connections 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 -- 172.21.15.98:0/1212116031 >> 172.21.15.98:0/1212116031 conn(0x7f63d8068100 msgr2=0x7f63d80684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 -- 172.21.15.98:0/1212116031 shutdown_connections 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.211+0000 7f63d7fff700 1 -- 172.21.15.98:0/1212116031 wait complete. 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.212+0000 7f63d7fff700 1 Processor -- start 2023-10-06T01:41:29.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.212+0000 7f63d7fff700 1 -- start start 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 0x7f63d815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80c8e80 0x7f63d80c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815ca00 con 0x7f63d80c9f60 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8150980 con 0x7f63c4004810 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80c7b90 con 0x7f63d80c8e80 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33780/0 (socket says 172.21.15.98:33780) 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 -- 172.21.15.98:0/3284581840 learned_addr learned my addr 172.21.15.98:0/3284581840 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63ce7fc700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80c8e80 0x7f63d80c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d6ffd700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 0x7f63d815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 -- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80c8e80 msgr2=0x7f63d80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80c8e80 0x7f63d80c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 -- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 msgr2=0x7f63d815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 0x7f63d815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 -- 172.21.15.98:0/3284581840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80c89f0 con 0x7f63d80c9f60 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d6ffd700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 0x7f63d815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d77fe700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009d70 tx=0x7f63d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e040 con 0x7f63d80c9f60 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d000f040 con 0x7f63d80c9f60 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000fdf0 con 0x7f63d80c9f60 2023-10-06T01:41:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d80c7de0 con 0x7f63d80c9f60 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.213+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d80c83a0 con 0x7f63d80c9f60 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.214+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d80c8b80 con 0x7f63d80c9f60 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.215+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63d0015640 con 0x7f63d80c9f60 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.215+0000 7f63cf7fe700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.215+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f63d0085270 con 0x7f63d80c9f60 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.215+0000 7f63d6ffd700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.215+0000 7f63d6ffd700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000fd0 tx=0x7f63c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.219+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63d0052020 con 0x7f63d80c9f60 2023-10-06T01:41:29.401 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:29.400+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63d805f270 con 0x7f63d80c9f60 2023-10-06T01:41:30.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.075+0000 7f63cf7fe700 1 -- 172.21.15.98:0/3284581840 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f63d0055640 con 0x7f63d80c9f60 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 msgr2=0x7f63b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000fd0 tx=0x7f63c8009040 comp rx=0 tx=0).stop 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 msgr2=0x7f63d81599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009d70 tx=0x7f63d000b040 comp rx=0 tx=0).stop 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 shutdown_connections 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63d80c9f60 0x7f63d81599d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c4004810 0x7f63d815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 --2- 172.21.15.98:0/3284581840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d80c8e80 0x7f63d80c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 >> 172.21.15.98:0/3284581840 conn(0x7f63d8068100 msgr2=0x7f63d80c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 shutdown_connections 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.078+0000 7f63d7fff700 1 -- 172.21.15.98:0/3284581840 wait complete. 2023-10-06T01:41:30.080 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2023-10-06T01:41:30.092 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T01:41:30.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.210+0000 7f4e2fa84700 1 Processor -- start 2023-10-06T01:41:30.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.210+0000 7f4e2fa84700 1 -- start start 2023-10-06T01:41:30.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2fa84700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4e280b23a0 0x7f4e280b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2fa84700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 0x7f4e280af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2fa84700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e280b2d20 con 0x7f4e2806ed20 2023-10-06T01:41:30.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2fa84700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2806fb80 con 0x7f4e280b23a0 2023-10-06T01:41:30.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2fa84700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e280b2840 con 0x7f4e280aee20 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4e280b23a0 0x7f4e280b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4e280b23a0 0x7f4e280b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33792/0 (socket says 172.21.15.98:33792) 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 -- 172.21.15.98:0/3740202932 learned_addr learned my addr 172.21.15.98:0/3740202932 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2e021700 1 --2- 172.21.15.98:0/3740202932 >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 0x7f4e280af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 -- 172.21.15.98:0/3740202932 >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 msgr2=0x7f4e280af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 --2- 172.21.15.98:0/3740202932 >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 0x7f4e280af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 -- 172.21.15.98:0/3740202932 >> v1:172.21.15.184:6789/0 conn(0x7f4e2806ed20 legacy=0x7f4e2806f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2d01f700 1 -- 172.21.15.98:0/3740202932 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280b0450 con 0x7f4e280b23a0 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.211+0000 7f4e2e021700 1 --2- 172.21.15.98:0/3740202932 >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 0x7f4e280af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:30.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2d01f700 1 --2- 172.21.15.98:0/3740202932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e280b08e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4e280b1640 tx=0x7f4e18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37224cfe7d82b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2c81e700 1 -- 172.21.15.98:0/3740202932 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800b860 con 0x7f4e280b23a0 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2c81e700 1 -- 172.21.15.98:0/3740202932 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1800b9c0 con 0x7f4e280b23a0 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2c81e700 1 -- 172.21.15.98:0/3740202932 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800bd60 con 0x7f4e280b23a0 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 -- 172.21.15.98:0/3740202932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 msgr2=0x7f4e280b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 --2- 172.21.15.98:0/3740202932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e280b08e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4e280b1640 tx=0x7f4e18009250 comp rx=0 tx=0).stop 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 -- 172.21.15.98:0/3740202932 shutdown_connections 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 --2- 172.21.15.98:0/3740202932 >> v2:172.21.15.184:3300/0 conn(0x7f4e280aee20 0x7f4e280af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 --2- 172.21.15.98:0/3740202932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e280b08e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 -- 172.21.15.98:0/3740202932 >> 172.21.15.98:0/3740202932 conn(0x7f4e28068330 msgr2=0x7f4e28068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:30.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 -- 172.21.15.98:0/3740202932 shutdown_connections 2023-10-06T01:41:30.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.212+0000 7f4e2fa84700 1 -- 172.21.15.98:0/3740202932 wait complete. 2023-10-06T01:41:30.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.213+0000 7f4e2fa84700 1 Processor -- start 2023-10-06T01:41:30.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.213+0000 7f4e2fa84700 1 -- start start 2023-10-06T01:41:30.218 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e2806ed20 0x7f4e2815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e280aee20 0x7f4e2816ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e28158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281562b0 con 0x7f4e280b23a0 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280a6fd0 con 0x7f4e280aee20 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2fa84700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2816c0f0 con 0x7f4e2806ed20 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2e021700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e28158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2e021700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e28158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33794/0 (socket says 172.21.15.98:33794) 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2e021700 1 -- 172.21.15.98:0/2666856837 learned_addr learned my addr 172.21.15.98:0/2666856837 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:30.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2d01f700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e280aee20 0x7f4e2816ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.214+0000 7f4e2d820700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e2806ed20 0x7f4e2815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 -- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e2806ed20 msgr2=0x7f4e2815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e2806ed20 0x7f4e2815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 -- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e280aee20 msgr2=0x7f4e2816ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e280aee20 0x7f4e2816ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 -- 172.21.15.98:0/2666856837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180094f0 con 0x7f4e280b23a0 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2d820700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e2806ed20 0x7f4e2815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2e021700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e280b23a0 0x7f4e28158700 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000d890 tx=0x7f4e20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200125f0 con 0x7f4e280b23a0 2023-10-06T01:41:30.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20012750 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2fa84700 1 -- 172.21.15.98:0/2666856837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2816b240 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20012ad0 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.215+0000 7f4e2fa84700 1 -- 172.21.15.98:0/2666856837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2816b7c0 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.216+0000 7f4e2fa84700 1 -- 172.21.15.98:0/2666856837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28064ad0 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.216+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e20014430 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.217+0000 7f4e1dffb700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4e0c066f30 0x7f4e0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.217+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f4e20084bb0 con 0x7f4e280b23a0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.217+0000 7f4e2d820700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4e0c066f30 0x7f4e0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.217+0000 7f4e2d820700 1 --2- 172.21.15.98:0/2666856837 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4e0c066f30 0x7f4e0c0693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e24007fd0 tx=0x7f4e24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.221+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e20051b20 con 0x7f4e280b23a0 2023-10-06T01:41:30.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.404+0000 7f4e2fa84700 1 -- 172.21.15.98:0/2666856837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4e2816c230 con 0x7f4e280b23a0 2023-10-06T01:41:30.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.404+0000 7f4e1dffb700 1 -- 172.21.15.98:0/2666856837 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=-16 pool 'cephfs_data' is in use by CephFS v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4e20055140 con 0x7f4e280b23a0 2023-10-06T01:41:30.406 INFO:teuthology.orchestra.run.smithi098.stderr:Error EBUSY: pool 'cephfs_data' is in use by CephFS 2023-10-06T01:41:30.409 DEBUG:teuthology.orchestra.run:got remote process result: 16 2023-10-06T01:41:30.410 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:30.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.528+0000 7fa708399700 1 Processor -- start 2023-10-06T01:41:30.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.528+0000 7fa708399700 1 -- start start 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa708399700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa7000ca3b0 0x7fa7001582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa708399700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa7000adba0 0x7fa7000ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa708399700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa700157e20 con 0x7fa7000ce710 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa708399700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000cbbd0 con 0x7fa7000adba0 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa708399700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000ceb50 con 0x7fa7000ca3b0 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.529+0000 7fa705934700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa7000adba0 0x7fa7000ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa7000adba0 0x7fa7000ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33798/0 (socket says 172.21.15.98:33798) 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 -- 172.21.15.98:0/2136120544 learned_addr learned my addr 172.21.15.98:0/2136120544 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa706135700 1 --2- 172.21.15.98:0/2136120544 >> v2:172.21.15.184:3300/0 conn(0x7fa7000ca3b0 0x7fa7001582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.532 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 -- 172.21.15.98:0/2136120544 >> v2:172.21.15.184:3300/0 conn(0x7fa7000ca3b0 msgr2=0x7fa7001582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 --2- 172.21.15.98:0/2136120544 >> v2:172.21.15.184:3300/0 conn(0x7fa7000ca3b0 0x7fa7001582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 -- 172.21.15.98:0/2136120544 >> v1:172.21.15.184:6789/0 conn(0x7fa7000ce710 legacy=0x7fa70015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 -- 172.21.15.98:0/2136120544 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70006a710 con 0x7fa7000adba0 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705934700 1 --2- 172.21.15.98:0/2136120544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000adba0 0x7fa7000ce1d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa70006f300 tx=0x7fa6fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8586e908a26b18d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705133700 1 -- 172.21.15.98:0/2136120544 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00b860 con 0x7fa7000adba0 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705133700 1 -- 172.21.15.98:0/2136120544 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc00b9c0 con 0x7fa7000adba0 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.530+0000 7fa705133700 1 -- 172.21.15.98:0/2136120544 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00bd60 con 0x7fa7000adba0 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 -- 172.21.15.98:0/2136120544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000adba0 msgr2=0x7fa7000ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 --2- 172.21.15.98:0/2136120544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000adba0 0x7fa7000ce1d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa70006f300 tx=0x7fa6fc009250 comp rx=0 tx=0).stop 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 -- 172.21.15.98:0/2136120544 shutdown_connections 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 --2- 172.21.15.98:0/2136120544 >> v2:172.21.15.184:3300/0 conn(0x7fa7000ca3b0 0x7fa7001582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.533 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 --2- 172.21.15.98:0/2136120544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000adba0 0x7fa7000ce1d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 -- 172.21.15.98:0/2136120544 >> 172.21.15.98:0/2136120544 conn(0x7fa700070810 msgr2=0x7fa700070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:30.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 -- 172.21.15.98:0/2136120544 shutdown_connections 2023-10-06T01:41:30.534 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.531+0000 7fa708399700 1 -- 172.21.15.98:0/2136120544 wait complete. 2023-10-06T01:41:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.532+0000 7fa708399700 1 Processor -- start 2023-10-06T01:41:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.532+0000 7fa708399700 1 -- start start 2023-10-06T01:41:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7000adba0 0x7fa7000b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.536 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 0x7fa700148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000b2c50 con 0x7fa7000ca3b0 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000b3770 con 0x7fa7000adba0 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa708399700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa700144760 con 0x7fa7000ce710 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33804/0 (socket says 172.21.15.98:33804) 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 -- 172.21.15.98:0/1207894752 learned_addr learned my addr 172.21.15.98:0/1207894752 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa706135700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7000adba0 0x7fa7000b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa706936700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 0x7fa700148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 -- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 msgr2=0x7fa700148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.537 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 0x7fa700148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 -- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7000adba0 msgr2=0x7fa7000b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7000adba0 0x7fa7000b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa705934700 1 -- 172.21.15.98:0/1207894752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0094f0 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.533+0000 7fa706936700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 0x7fa700148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa705934700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006fd0 tx=0x7fa6fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc011e20 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc01a630 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7001442a0 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc01a9f0 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.534+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7000b3120 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.535+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa700064ad0 con 0x7fa7000ca3b0 2023-10-06T01:41:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.535+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6fc02d440 con 0x7fa7000ca3b0 2023-10-06T01:41:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.535+0000 7fa6f67fc700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 0x7fa6e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.536+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7fa6fc086310 con 0x7fa7000ca3b0 2023-10-06T01:41:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.536+0000 7fa706135700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 0x7fa6e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.539 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.536+0000 7fa706135700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 0x7fa6e40693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8007460 tx=0x7fa6f800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.540+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6fc053280 con 0x7fa7000ca3b0 2023-10-06T01:41:30.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.725+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7000cb610 con 0x7fa7000ca3b0 2023-10-06T01:41:30.726 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.726+0000 7fa6f67fc700 1 -- 172.21.15.98:0/1207894752 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+17313 (secure 0 0 0) 0x7fa6fc0568a0 con 0x7fa7000ca3b0 2023-10-06T01:41:30.727 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:30.727 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":45,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:30.073742+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 msgr2=0x7fa6e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 0x7fa6e40693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8007460 tx=0x7fa6f800b040 comp rx=0 tx=0).stop 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 msgr2=0x7fa700146c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006fd0 tx=0x7fa6fc017000 comp rx=0 tx=0).stop 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 shutdown_connections 2023-10-06T01:41:30.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa6e4066f30 0x7fa6e40693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7000ce710 0x7fa700148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7000adba0 0x7fa7000b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 --2- 172.21.15.98:0/1207894752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7000ca3b0 0x7fa700146c50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 >> 172.21.15.98:0/1207894752 conn(0x7fa700070810 msgr2=0x7fa700155530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 shutdown_connections 2023-10-06T01:41:30.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.728+0000 7fa708399700 1 -- 172.21.15.98:0/1207894752 wait complete. 2023-10-06T01:41:30.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T01:39:37.371756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T01:39:37.826565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T01:40:40.800463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}}, {'pool': 8, 'pool_name': 'recovery_fs_metadata', 'create_time': '2023-10-06T01:41:03.494206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}}] 2023-10-06T01:41:30.741 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T01:41:30.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.857+0000 7fc183fff700 1 Processor -- start 2023-10-06T01:41:30.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.857+0000 7fc183fff700 1 -- start start 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc183fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1840b2a30 0x7fc184164360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc183fff700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1840cc9b0 con 0x7fc1840cd400 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc183fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1840cbda0 con 0x7fc1840b2660 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc183fff700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840cc080 con 0x7fc1840b2a30 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1837fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1840b2a30 0x7fc184164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc17a7fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc1840cd400 0x7fc1840ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52150/0 (socket says 172.21.15.98:52150) 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc17a7fc700 1 -- 172.21.15.98:0/2431828333 learned_addr learned my addr 172.21.15.98:0/2431828333 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1837fe700 1 -- 172.21.15.98:0/2431828333 >> v1:172.21.15.184:6789/0 conn(0x7fc1840b2660 legacy=0x7fc1840cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1837fe700 1 -- 172.21.15.98:0/2431828333 >> v1:172.21.15.98:6789/0 conn(0x7fc1840cd400 legacy=0x7fc1840ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1837fe700 1 -- 172.21.15.98:0/2431828333 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840cc360 con 0x7fc1840b2a30 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1827fc700 1 -- 172.21.15.98:0/2431828333 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573795207 0 0) 0x7fc1840cc9b0 con 0x7fc1840cd400 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.858+0000 7fc1837fe700 1 --2- 172.21.15.98:0/2431828333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2a30 0x7fc184164360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e60 tx=0x7fc17c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8159c1d652282f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc1827fc700 1 -- 172.21.15.98:0/2431828333 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00e070 con 0x7fc1840b2a30 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc1827fc700 1 -- 172.21.15.98:0/2431828333 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00bbf0 con 0x7fc1840b2a30 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc1827fc700 1 -- 172.21.15.98:0/2431828333 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c012430 con 0x7fc1840b2a30 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 -- 172.21.15.98:0/2431828333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2a30 msgr2=0x7fc184164360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 --2- 172.21.15.98:0/2431828333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2a30 0x7fc184164360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e60 tx=0x7fc17c009780 comp rx=0 tx=0).stop 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 -- 172.21.15.98:0/2431828333 shutdown_connections 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 --2- 172.21.15.98:0/2431828333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2a30 0x7fc184164360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 -- 172.21.15.98:0/2431828333 >> 172.21.15.98:0/2431828333 conn(0x7fc184099560 msgr2=0x7fc18406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 -- 172.21.15.98:0/2431828333 shutdown_connections 2023-10-06T01:41:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.859+0000 7fc183fff700 1 -- 172.21.15.98:0/2431828333 wait complete. 2023-10-06T01:41:30.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.860+0000 7fc183fff700 1 Processor -- start 2023-10-06T01:41:30.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.860+0000 7fc183fff700 1 -- start start 2023-10-06T01:41:30.864 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1840b2a30 0x7fc184147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1840cd400 0x7fc184157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18416dc50 con 0x7fc1840b2660 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840b4a50 con 0x7fc1840cd400 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc183fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840c8270 con 0x7fc1840b2a30 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33824/0 (socket says 172.21.15.98:33824) 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 -- 172.21.15.98:0/3748208776 learned_addr learned my addr 172.21.15.98:0/3748208776 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc1837fe700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1840cd400 0x7fc184157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc17a7fc700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1840b2a30 0x7fc184147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.865 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 -- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1840b2a30 msgr2=0x7fc184147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1840b2a30 0x7fc184147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 -- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1840cd400 msgr2=0x7fc184157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1840cd400 0x7fc184157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.861+0000 7fc182ffd700 1 -- 172.21.15.98:0/3748208776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009a20 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc182ffd700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc17400b5e0 tx=0x7fc1740090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400f510 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400f670 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840c8550 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400fa30 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.862+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1840c8b10 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.863+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18405f270 con 0x7fc1840b2660 2023-10-06T01:41:30.866 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.863+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc17401a400 con 0x7fc1840b2660 2023-10-06T01:41:30.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.863+0000 7fc17b7fe700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:30.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.864+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7fc1740842d0 con 0x7fc1840b2660 2023-10-06T01:41:30.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.864+0000 7fc17a7fc700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:30.867 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.864+0000 7fc17a7fc700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1840b4870 tx=0x7fc16c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:30.868 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:30.868+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc174054d50 con 0x7fc1840b2660 2023-10-06T01:41:31.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.048+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc1840cc6d0 con 0x7fc1840b2660 2023-10-06T01:41:31.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.048+0000 7fc17b7fe700 1 -- 172.21.15.98:0/3748208776 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1740548e0 con 0x7fc1840b2660 2023-10-06T01:41:31.049 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 8 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 msgr2=0x7fc1640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1840b4870 tx=0x7fc16c009040 comp rx=0 tx=0).stop 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 msgr2=0x7fc18416d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc17400b5e0 tx=0x7fc1740090f0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 shutdown_connections 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1840cd400 0x7fc184157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1840b2660 0x7fc18416d710 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 --2- 172.21.15.98:0/3748208776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1840b2a30 0x7fc184147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 >> 172.21.15.98:0/3748208776 conn(0x7fc184099560 msgr2=0x7fc184069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 shutdown_connections 2023-10-06T01:41:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.050+0000 7fc183fff700 1 -- 172.21.15.98:0/3748208776 wait complete. 2023-10-06T01:41:31.062 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T01:41:31.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.179+0000 7f7056bb5700 1 Processor -- start 2023-10-06T01:41:31.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.179+0000 7f7056bb5700 1 -- start start 2023-10-06T01:41:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7056bb5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7056bb5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70501598e0 0x7f70500cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7056bb5700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70500cf350 con 0x7f70500ce820 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7056bb5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70500cf610 con 0x7f70501598e0 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7056bb5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70500cf8f0 con 0x7f7050159510 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f704ffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70501598e0 0x7f70500cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7055152700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f70500ce820 0x7f70500cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52160/0 (socket says 172.21.15.98:52160) 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f704ffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70501598e0 0x7f70500cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33838/0 (socket says 172.21.15.98:33838) 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7055152700 1 -- 172.21.15.98:0/238883327 learned_addr learned my addr 172.21.15.98:0/238883327 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.180+0000 7f7054951700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704ffff700 1 -- 172.21.15.98:0/238883327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 msgr2=0x7f70500cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704ffff700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704f7fe700 1 -- 172.21.15.98:0/238883327 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198571927 0 0) 0x7f70500cf350 con 0x7f70500ce820 2023-10-06T01:41:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704ffff700 1 -- 172.21.15.98:0/238883327 >> v1:172.21.15.98:6789/0 conn(0x7f70500ce820 legacy=0x7f70500cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704ffff700 1 -- 172.21.15.98:0/238883327 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500cfbd0 con 0x7f70501598e0 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704ffff700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70501598e0 0x7f70500cbae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f70440098e0 tx=0x7f70440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f59b54537d352e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704f7fe700 1 -- 172.21.15.98:0/238883327 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044009e80 con 0x7f70501598e0 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704f7fe700 1 -- 172.21.15.98:0/238883327 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704400b860 con 0x7f70501598e0 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f704f7fe700 1 -- 172.21.15.98:0/238883327 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704400bbe0 con 0x7f70501598e0 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 -- 172.21.15.98:0/238883327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70501598e0 msgr2=0x7f70500cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70501598e0 0x7f70500cbae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f70440098e0 tx=0x7f70440091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 -- 172.21.15.98:0/238883327 shutdown_connections 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 --2- 172.21.15.98:0/238883327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70501598e0 0x7f70500cbae0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 -- 172.21.15.98:0/238883327 >> 172.21.15.98:0/238883327 conn(0x7f7050089e90 msgr2=0x7f70500997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 -- 172.21.15.98:0/238883327 shutdown_connections 2023-10-06T01:41:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.181+0000 7f7056bb5700 1 -- 172.21.15.98:0/238883327 wait complete. 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.182+0000 7f7056bb5700 1 Processor -- start 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 -- start start 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 0x7f70500c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050156bc0 con 0x7f70500ce820 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705016a720 con 0x7f70501598e0 2023-10-06T01:41:31.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7056bb5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705016aa00 con 0x7f7050159510 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7054951700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.183+0000 7f7054951700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33850/0 (socket says 172.21.15.98:33850) 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 -- 172.21.15.98:0/3144793482 learned_addr learned my addr 172.21.15.98:0/3144793482 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f704ffff700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7055152700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 0x7f70500c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 -- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 msgr2=0x7f70500c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 -- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 msgr2=0x7f70500c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 0x7f70500c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 -- 172.21.15.98:0/3144793482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044009420 con 0x7f70500ce820 2023-10-06T01:41:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7055152700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 0x7f70500c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f704ffff700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7054951700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f704000e190 tx=0x7f7040009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000ee30 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040014420 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7050156dd0 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400147e0 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.184+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705015a840 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.185+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705015cfe0 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.186+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7040014b90 con 0x7f70500ce820 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.186+0000 7f704cff9700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 0x7f703c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.186+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f7040085030 con 0x7f70500ce820 2023-10-06T01:41:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.186+0000 7f704ffff700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 0x7f703c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.186+0000 7f704ffff700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 0x7f703c069410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7050159cd0 tx=0x7f7044017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.190+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7040051fa0 con 0x7f70500ce820 2023-10-06T01:41:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.373+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7050064ad0 con 0x7f70500ce820 2023-10-06T01:41:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.374+0000 7f704cff9700 1 -- 172.21.15.98:0/3144793482 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7040009a60 con 0x7f70500ce820 2023-10-06T01:41:31.374 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2023-10-06T01:41:31.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 msgr2=0x7f703c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 0x7f703c069410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7050159cd0 tx=0x7f7044017000 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 msgr2=0x7f705014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f704000e190 tx=0x7f7040009920 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 shutdown_connections 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f703c066f60 0x7f703c069410 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70501598e0 0x7f70500c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70500ce820 0x7f705014b500 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.375+0000 7f7056bb5700 1 --2- 172.21.15.98:0/3144793482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7050159510 0x7f70500c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.376+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 >> 172.21.15.98:0/3144793482 conn(0x7f7050089e90 msgr2=0x7f705015d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.376+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 shutdown_connections 2023-10-06T01:41:31.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.376+0000 7f7056bb5700 1 -- 172.21.15.98:0/3144793482 wait complete. 2023-10-06T01:41:31.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T01:41:31.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.505+0000 7f5ee4f27700 1 Processor -- start 2023-10-06T01:41:31.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.505+0000 7f5ee4f27700 1 -- start start 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ee4f27700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ee00cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ee4f27700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00cc9b0 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ee4f27700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00cbda0 con 0x7f5ee00b2a30 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ee4f27700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00cc080 con 0x7f5ee00b2660 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5edffff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5ee00cf1f0 0x7f5ee00c5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52176/0 (socket says 172.21.15.98:52176) 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5edffff700 1 -- 172.21.15.98:0/667312710 learned_addr learned my addr 172.21.15.98:0/667312710 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ee00cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500002265 0 0) 0x7f5ee00cc9b0 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.506+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec8003570 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063528793 0 0) 0x7f5ee00cbda0 con 0x7f5ee00b2a30 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee00cc9b0 con 0x7f5ee00b2a30 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577476402 0 0) 0x7f5ec8003570 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 msgr2=0x7f5ee00cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ee00cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 >> v1:172.21.15.184:6789/0 conn(0x7f5ee00b2a30 legacy=0x7f5ee00ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00cc360 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f5ed80017f0 con 0x7f5ee00cf1f0 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 >> v1:172.21.15.98:6789/0 conn(0x7f5ee00cf1f0 legacy=0x7f5ee00c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec8003c20 0x7f5ec8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8004810 0x7f5ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00cc080 con 0x7f5ec8003c20 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec800bb40 con 0x7f5ec8004810 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec800bd00 con 0x7f5ee00b2660 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec8003c20 0x7f5ec8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edeffd700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8004810 0x7f5ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edffff700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 -- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 msgr2=0x7f5ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee00b2660 0x7f5ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 -- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8004810 msgr2=0x7f5ec8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8004810 0x7f5ec8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 -- 172.21.15.98:0/667312710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8003570 con 0x7f5ec8003c20 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec8003c20 0x7f5ec8004010 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009bf0 tx=0x7f5ed0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc5afe8f82cdc6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0005a00 con 0x7f5ec8003c20 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed0005b60 con 0x7f5ec8003c20 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.507+0000 7f5ede7fc700 1 -- 172.21.15.98:0/667312710 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed000e710 con 0x7f5ec8003c20 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 -- 172.21.15.98:0/667312710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec8003c20 msgr2=0x7f5ec8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/667312710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec8003c20 0x7f5ec8004010 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009bf0 tx=0x7f5ed0009120 comp rx=0 tx=0).stop 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5edffff700 1 -- 172.21.15.98:0/667312710 reap_dead start 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 -- 172.21.15.98:0/667312710 shutdown_connections 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 -- 172.21.15.98:0/667312710 >> 172.21.15.98:0/667312710 conn(0x7f5ee0099560 msgr2=0x7f5ee006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 -- 172.21.15.98:0/667312710 shutdown_connections 2023-10-06T01:41:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.508+0000 7f5ee4f27700 1 -- 172.21.15.98:0/667312710 wait complete. 2023-10-06T01:41:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.509+0000 7f5ee4f27700 1 Processor -- start 2023-10-06T01:41:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.509+0000 7f5ee4f27700 1 -- start start 2023-10-06T01:41:31.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 0x7f5ee00b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 0x7f5ee015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.516 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00c7d70 con 0x7f5ee00c73c0 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee015f770 con 0x7f5ec8003c20 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee015fa50 con 0x7f5ee015ecd0 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33868/0 (socket says 172.21.15.98:33868) 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 -- 172.21.15.98:0/1155317659 learned_addr learned my addr 172.21.15.98:0/1155317659 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edeffd700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 0x7f5ee015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 0x7f5ee00b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 -- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 msgr2=0x7f5ee015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 0x7f5ee015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 -- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 msgr2=0x7f5ee00b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 0x7f5ee00b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 -- 172.21.15.98:0/1155317659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00b39f0 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 0x7f5ee00b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edeffd700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 0x7f5ee015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5edffff700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00cf5c0 tx=0x7f5ed8001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8007b30 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed800ebc0 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee00b3cd0 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8013420 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.510+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee0153450 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.511+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee00b3850 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.512+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ed8013790 con 0x7f5ee00c73c0 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.512+0000 7f5ec77fe700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.512+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.512+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f5ed8084c90 con 0x7f5ee00c73c0 2023-10-06T01:41:31.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.513+0000 7f5edf7fe700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 0x7f5ecc0693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009160 tx=0x7f5ed0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.517+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ee00b3850 con 0x7f5ee00c73c0 2023-10-06T01:41:31.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.698+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ee00b3850 con 0x7f5ee00c73c0 2023-10-06T01:41:31.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.699+0000 7f5ec77fe700 1 -- 172.21.15.98:0/1155317659 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ee00b3850 con 0x7f5ee00c73c0 2023-10-06T01:41:31.699 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 msgr2=0x7f5ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 0x7f5ecc0693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009160 tx=0x7f5ed0019000 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 msgr2=0x7f5ee00c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00cf5c0 tx=0x7f5ed8001090 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 shutdown_connections 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ee00c73c0 0x7f5ee00c7830 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec8003c20 0x7f5ee00b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 --2- 172.21.15.98:0/1155317659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ee015ecd0 0x7f5ee015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 >> 172.21.15.98:0/1155317659 conn(0x7f5ee0099560 msgr2=0x7f5ee0068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 shutdown_connections 2023-10-06T01:41:31.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.701+0000 7f5ee4f27700 1 -- 172.21.15.98:0/1155317659 wait complete. 2023-10-06T01:41:31.713 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2023-10-06T01:41:31.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.833+0000 7f8e60d44700 1 Processor -- start 2023-10-06T01:41:31.834 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.833+0000 7f8e60d44700 1 -- start start 2023-10-06T01:41:31.836 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e60d44700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e5c159510 0x7f8e5c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e60d44700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e60d44700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c0cf350 con 0x7f8e5c0ce820 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e60d44700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0cf610 con 0x7f8e5c159510 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e60d44700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0cf8f0 con 0x7f8e5c1598e0 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e5c159510 0x7f8e5c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e5c159510 0x7f8e5c0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33874/0 (socket says 172.21.15.98:33874) 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 -- 172.21.15.98:0/3703316680 learned_addr learned my addr 172.21.15.98:0/3703316680 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e51d9b700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.837 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 -- 172.21.15.98:0/3703316680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 msgr2=0x7f8e5c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 -- 172.21.15.98:0/3703316680 >> v1:172.21.15.184:6789/0 conn(0x7f8e5c0ce820 legacy=0x7f8e5c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.834+0000 7f8e5a59c700 1 -- 172.21.15.98:0/3703316680 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0cfbd0 con 0x7f8e5c159510 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e5a59c700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c0cb5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0099b0 tx=0x7f8e4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc69ed42eb0b50f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e59d9b700 1 -- 172.21.15.98:0/3703316680 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00b860 con 0x7f8e5c159510 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e59d9b700 1 -- 172.21.15.98:0/3703316680 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c00b9c0 con 0x7f8e5c159510 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e59d9b700 1 -- 172.21.15.98:0/3703316680 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00bd60 con 0x7f8e5c159510 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 -- 172.21.15.98:0/3703316680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 msgr2=0x7f8e5c0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c0cb5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0099b0 tx=0x7f8e4c009250 comp rx=0 tx=0).stop 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 -- 172.21.15.98:0/3703316680 shutdown_connections 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c0cb5a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 --2- 172.21.15.98:0/3703316680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 -- 172.21.15.98:0/3703316680 >> 172.21.15.98:0/3703316680 conn(0x7f8e5c089e90 msgr2=0x7f8e5c0997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 -- 172.21.15.98:0/3703316680 shutdown_connections 2023-10-06T01:41:31.839 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.835+0000 7f8e60d44700 1 -- 172.21.15.98:0/3703316680 wait complete. 2023-10-06T01:41:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.836+0000 7f8e60d44700 1 Processor -- start 2023-10-06T01:41:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 -- start start 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e5c0ce820 0x7f8e5c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c142400 con 0x7f8e5c159510 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0c9000 con 0x7f8e5c0ce820 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e60d44700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0c83f0 con 0x7f8e5c1598e0 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e51d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e51d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33884/0 (socket says 172.21.15.98:33884) 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.837+0000 7f8e51d9b700 1 -- 172.21.15.98:0/1820123241 learned_addr learned my addr 172.21.15.98:0/1820123241 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e5a59c700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e5c0ce820 0x7f8e5c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e5ad9d700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 -- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 msgr2=0x7f8e5c141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 -- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e5c0ce820 msgr2=0x7f8e5c0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e5c0ce820 0x7f8e5c0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 -- 172.21.15.98:0/1820123241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0094f0 con 0x7f8e5c159510 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e5ad9d700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e51d9b700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006b00 tx=0x7f8e44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440125f0 con 0x7f8e5c159510 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44012770 con 0x7f8e5c159510 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0c86d0 con 0x7f8e5c159510 2023-10-06T01:41:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44012b30 con 0x7f8e5c159510 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.838+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c0c8c90 con 0x7f8e5c159510 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.839+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0c8e20 con 0x7f8e5c159510 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.839+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e4401e460 con 0x7f8e5c159510 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.840+0000 7f8e52ffd700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 0x7f8e3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.840+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f8e44084e20 con 0x7f8e5c159510 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.840+0000 7f8e5a59c700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 0x7f8e3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.840+0000 7f8e5a59c700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 0x7f8e3c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0099b0 tx=0x7f8e4c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:31.844 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:31.844+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e44051d90 con 0x7f8e5c159510 2023-10-06T01:41:32.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.024+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e5c064ad0 con 0x7f8e5c159510 2023-10-06T01:41:32.025 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.024+0000 7f8e52ffd700 1 -- 172.21.15.98:0/1820123241 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 109+0+11 (secure 0 0 0) 0x7f8e440553b0 con 0x7f8e5c159510 2023-10-06T01:41:32.025 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.026+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 msgr2=0x7f8e3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.026+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 0x7f8e3c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0099b0 tx=0x7f8e4c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.026+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 msgr2=0x7f8e5c141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.026+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006b00 tx=0x7f8e44008040 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 shutdown_connections 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8e3c066f30 0x7f8e3c0693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e5c1598e0 0x7f8e5c141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e5c0ce820 0x7f8e5c0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 --2- 172.21.15.98:0/1820123241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e5c159510 0x7f8e5c141980 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 >> 172.21.15.98:0/1820123241 conn(0x7f8e5c089e90 msgr2=0x7f8e5c0c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 shutdown_connections 2023-10-06T01:41:32.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.027+0000 7f8e60d44700 1 -- 172.21.15.98:0/1820123241 wait complete. 2023-10-06T01:41:32.039 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:32.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.153+0000 7eff66311700 1 Processor -- start 2023-10-06T01:41:32.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.153+0000 7eff66311700 1 -- start start 2023-10-06T01:41:32.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff601595a0 0x7eff600aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff600b49e0 0x7eff600aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7eff600b07a0 0x7eff600b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.156 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff600b0ca0 con 0x7eff600b07a0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff600b0f80 con 0x7eff600b49e0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff66311700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600b1260 con 0x7eff601595a0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7eff600b07a0 0x7eff600b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7eff600b07a0 0x7eff600b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33892/0 (socket says 172.21.15.98:33892) 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 -- 172.21.15.98:0/3192708828 learned_addr learned my addr 172.21.15.98:0/3192708828 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff5ffff700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff601595a0 0x7eff600aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff5f7fe700 1 --2- 172.21.15.98:0/3192708828 >> v2:172.21.15.184:3300/0 conn(0x7eff600b49e0 0x7eff600aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 -- 172.21.15.98:0/3192708828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff601595a0 msgr2=0x7eff600aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff601595a0 0x7eff600aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.157 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 -- 172.21.15.98:0/3192708828 >> v2:172.21.15.184:3300/0 conn(0x7eff600b49e0 msgr2=0x7eff600aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 --2- 172.21.15.98:0/3192708828 >> v2:172.21.15.184:3300/0 conn(0x7eff600b49e0 0x7eff600aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.154+0000 7eff648ae700 1 -- 172.21.15.98:0/3192708828 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600b1540 con 0x7eff600b07a0 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff648ae700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff600b38d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7eff60089060 tx=0x7eff58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cae7aaab3366c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff5effd700 1 -- 172.21.15.98:0/3192708828 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800e070 con 0x7eff600b07a0 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff5effd700 1 -- 172.21.15.98:0/3192708828 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5800ba50 con 0x7eff600b07a0 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff5effd700 1 -- 172.21.15.98:0/3192708828 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff580092e0 con 0x7eff600b07a0 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 -- 172.21.15.98:0/3192708828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 msgr2=0x7eff600b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff600b38d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7eff60089060 tx=0x7eff58009840 comp rx=0 tx=0).stop 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 -- 172.21.15.98:0/3192708828 shutdown_connections 2023-10-06T01:41:32.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff600b38d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 --2- 172.21.15.98:0/3192708828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff601595a0 0x7eff600aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 --2- 172.21.15.98:0/3192708828 >> v2:172.21.15.184:3300/0 conn(0x7eff600b49e0 0x7eff600aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 -- 172.21.15.98:0/3192708828 >> 172.21.15.98:0/3192708828 conn(0x7eff60099830 msgr2=0x7eff6006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 -- 172.21.15.98:0/3192708828 shutdown_connections 2023-10-06T01:41:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.155+0000 7eff66311700 1 -- 172.21.15.98:0/3192708828 wait complete. 2023-10-06T01:41:32.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.156+0000 7eff66311700 1 Processor -- start 2023-10-06T01:41:32.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 -- start start 2023-10-06T01:41:32.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600b49e0 0x7eff600c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff601595a0 0x7eff600c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600c7260 con 0x7eff600b07a0 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff601580c0 con 0x7eff601595a0 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff66311700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600c5b70 con 0x7eff600b49e0 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33896/0 (socket says 172.21.15.98:33896) 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.157+0000 7eff5ffff700 1 -- 172.21.15.98:0/2934814867 learned_addr learned my addr 172.21.15.98:0/2934814867 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff648ae700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff601595a0 0x7eff600c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5f7fe700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600b49e0 0x7eff600c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 -- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600b49e0 msgr2=0x7eff600c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600b49e0 0x7eff600c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 -- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff601595a0 msgr2=0x7eff600c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff601595a0 0x7eff600c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 -- 172.21.15.98:0/2934814867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009ae0 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff5ffff700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7eff5000d840 tx=0x7eff50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50014620 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff50014780 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff600c5e50 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50014b40 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.158+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600c6ef0 con 0x7eff600b07a0 2023-10-06T01:41:32.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.159+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff60064ad0 con 0x7eff600b07a0 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.159+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff5001d400 con 0x7eff600b07a0 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.160+0000 7eff47fff700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.160+0000 7eff5f7fe700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.160+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7eff500850d0 con 0x7eff600b07a0 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.160+0000 7eff5f7fe700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 0x7eff4c0693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff6016ab10 tx=0x7eff54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.164+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff50052040 con 0x7eff600b07a0 2023-10-06T01:41:32.389 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.389+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff600b1820 con 0x7eff600b07a0 2023-10-06T01:41:32.390 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.389+0000 7eff47fff700 1 -- 172.21.15.98:0/2934814867 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4606 (secure 0 0 0) 0x7eff50055660 con 0x7eff600b07a0 2023-10-06T01:41:32.390 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:32.390 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:32.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 msgr2=0x7eff4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.392 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 0x7eff4c0693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff6016ab10 tx=0x7eff54008040 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 msgr2=0x7eff60157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7eff5000d840 tx=0x7eff50008040 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 shutdown_connections 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff601595a0 0x7eff600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff600b07a0 0x7eff60157b80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 --2- 172.21.15.98:0/2934814867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600b49e0 0x7eff600c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.391+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 >> 172.21.15.98:0/2934814867 conn(0x7eff60099830 msgr2=0x7eff60069610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.392+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 shutdown_connections 2023-10-06T01:41:32.393 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.392+0000 7eff66311700 1 -- 172.21.15.98:0/2934814867 wait complete. 2023-10-06T01:41:32.394 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 29 2023-10-06T01:41:32.404 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:32.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.517+0000 7f3be20f0700 1 Processor -- start 2023-10-06T01:41:32.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.517+0000 7f3be20f0700 1 -- start start 2023-10-06T01:41:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.518+0000 7f3be20f0700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bdc1595a0 0x7f3bdc0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.518+0000 7f3be20f0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.518+0000 7f3be20f0700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0b0ca0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.518+0000 7f3be20f0700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0b0f80 con 0x7f3bdc1595a0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.518+0000 7f3be20f0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0b1260 con 0x7f3bdc0b07a0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bdc1595a0 0x7f3bdc0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bdc1595a0 0x7f3bdc0aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41198/0 (socket says 172.21.15.98:41198) 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 -- 172.21.15.98:0/1972490185 learned_addr learned my addr 172.21.15.98:0/1972490185 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdbfff700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 -- 172.21.15.98:0/1972490185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 msgr2=0x7f3bdc0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 -- 172.21.15.98:0/1972490185 >> v1:172.21.15.184:6789/0 conn(0x7f3bdc0b49e0 legacy=0x7f3bdc0aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 -- 172.21.15.98:0/1972490185 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0b1540 con 0x7f3bdc1595a0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bdb7fe700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc0aa570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009fa0 tx=0x7f3bcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8359420224966cda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bda7fc700 1 -- 172.21.15.98:0/1972490185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00e070 con 0x7f3bdc1595a0 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bda7fc700 1 -- 172.21.15.98:0/1972490185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00bac0 con 0x7f3bdc1595a0 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.519+0000 7f3bda7fc700 1 -- 172.21.15.98:0/1972490185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0092e0 con 0x7f3bdc1595a0 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 -- 172.21.15.98:0/1972490185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc1595a0 msgr2=0x7f3bdc0aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc0aa570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009fa0 tx=0x7f3bcc009840 comp rx=0 tx=0).stop 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 -- 172.21.15.98:0/1972490185 shutdown_connections 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 --2- 172.21.15.98:0/1972490185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc0aa570 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 -- 172.21.15.98:0/1972490185 >> 172.21.15.98:0/1972490185 conn(0x7f3bdc099830 msgr2=0x7f3bdc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 -- 172.21.15.98:0/1972490185 shutdown_connections 2023-10-06T01:41:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.520+0000 7f3be20f0700 1 -- 172.21.15.98:0/1972490185 wait complete. 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.521+0000 7f3be20f0700 1 Processor -- start 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.521+0000 7f3be20f0700 1 -- start start 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc09a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc09cf70 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc09c6b0 con 0x7f3bdc1595a0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0aebb0 con 0x7f3bdc0b07a0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41200/0 (socket says 172.21.15.98:41200) 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 -- 172.21.15.98:0/3117732132 learned_addr learned my addr 172.21.15.98:0/3117732132 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdb7fe700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc09a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdbfff700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 -- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 msgr2=0x7f3bdc09a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc09a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 -- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bdc1595a0 msgr2=0x7f3bdc157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 -- 172.21.15.98:0/3117732132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009ae0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bdaffd700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000c7c0 tx=0x7f3bd0007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0011590 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd0011710 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc09c920 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0011ad0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.522+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc0aee80 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.523+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc09d0e0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.524+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bd001b3b0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.524+0000 7f3bc37fe700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 0x7f3bc8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.524+0000 7f3bdb7fe700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 0x7f3bc8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.524+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f3bd0083ce0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.525+0000 7f3bdb7fe700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 0x7f3bc8069410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009880 tx=0x7f3bcc018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.529 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.528+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bd0050c50 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.715 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.714+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bdc064ad0 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.716 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.715+0000 7f3bc37fe700 1 -- 172.21.15.98:0/3117732132 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+17313 (secure 0 0 0) 0x7f3bd0017070 con 0x7f3bdc0b49e0 2023-10-06T01:41:32.716 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:32.716 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":45,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:30.073742+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 msgr2=0x7f3bc8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 0x7f3bc8069410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009880 tx=0x7f3bcc018000 comp rx=0 tx=0).stop 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 msgr2=0x7f3bdc0ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000c7c0 tx=0x7f3bd0007f20 comp rx=0 tx=0).stop 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 shutdown_connections 2023-10-06T01:41:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3bc8066f60 0x7f3bc8069410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bdc1595a0 0x7f3bdc157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bdc0b07a0 0x7f3bdc09a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 --2- 172.21.15.98:0/3117732132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bdc0b49e0 0x7f3bdc0ae670 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 >> 172.21.15.98:0/3117732132 conn(0x7f3bdc099830 msgr2=0x7f3bdc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 shutdown_connections 2023-10-06T01:41:32.719 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.717+0000 7f3be20f0700 1 -- 172.21.15.98:0/3117732132 wait complete. 2023-10-06T01:41:32.730 INFO:tasks.cephfs.filesystem:Destroying file system recovery_fs and related pools 2023-10-06T01:41:32.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:32.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.844+0000 7f8783189700 1 Processor -- start 2023-10-06T01:41:32.845 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.844+0000 7f8783189700 1 -- start start 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8783189700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 0x7f877c0ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8783189700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f877c157830 0x7f877c0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8783189700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0cae60 con 0x7f877c0ac480 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8783189700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0aeed0 con 0x7f877c157830 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8783189700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0af1b0 con 0x7f877c157460 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f877bfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f877c157830 0x7f877c0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8781726700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f877c0ac480 0x7f877c068600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47862/0 (socket says 172.21.15.98:47862) 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.845+0000 7f8781726700 1 -- 172.21.15.98:0/2037422583 learned_addr learned my addr 172.21.15.98:0/2037422583 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8780f25700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 0x7f877c0ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877bfff700 1 -- 172.21.15.98:0/2037422583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 msgr2=0x7f877c0ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877bfff700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 0x7f877c0ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877bfff700 1 -- 172.21.15.98:0/2037422583 >> v1:172.21.15.98:6789/0 conn(0x7f877c0ac480 legacy=0x7f877c068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877b7fe700 1 -- 172.21.15.98:0/2037422583 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218991762 0 0) 0x7f877c0cae60 con 0x7f877c0ac480 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877bfff700 1 -- 172.21.15.98:0/2037422583 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0af490 con 0x7f877c157830 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8780f25700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 0x7f877c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877bfff700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c0cc2a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f87680098e0 tx=0x7f87680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60688f7e5fa89f8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877b7fe700 1 -- 172.21.15.98:0/2037422583 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768009e80 con 0x7f877c157830 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877b7fe700 1 -- 172.21.15.98:0/2037422583 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800b860 con 0x7f877c157830 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f877b7fe700 1 -- 172.21.15.98:0/2037422583 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800bbe0 con 0x7f877c157830 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8783189700 1 -- 172.21.15.98:0/2037422583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 msgr2=0x7f877c0cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8783189700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c0cc2a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f87680098e0 tx=0x7f87680091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8783189700 1 -- 172.21.15.98:0/2037422583 shutdown_connections 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.846+0000 7f8783189700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c157460 0x7f877c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.847+0000 7f8783189700 1 --2- 172.21.15.98:0/2037422583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c0cc2a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.847+0000 7f8783189700 1 -- 172.21.15.98:0/2037422583 >> 172.21.15.98:0/2037422583 conn(0x7f877c06de80 msgr2=0x7f877c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.847+0000 7f8783189700 1 -- 172.21.15.98:0/2037422583 shutdown_connections 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.847+0000 7f8783189700 1 -- 172.21.15.98:0/2037422583 wait complete. 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 Processor -- start 2023-10-06T01:41:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 -- start start 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c0ac480 0x7f877c0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877c157460 0x7f877c0a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0aa3e0 con 0x7f877c157830 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0aa610 con 0x7f877c157460 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.848+0000 7f8783189700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0aa8f0 con 0x7f877c0ac480 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41224/0 (socket says 172.21.15.98:41224) 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 -- 172.21.15.98:0/1592401862 learned_addr learned my addr 172.21.15.98:0/1592401862 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:32.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 -- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c0ac480 msgr2=0x7f877c0ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c0ac480 0x7f877c0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 -- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877c157460 msgr2=0x7f877c0a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f877bfff700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877c157460 0x7f877c0a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877c157460 0x7f877c0a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 -- 172.21.15.98:0/1592401862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768009420 con 0x7f877c157830 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8781726700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f877400cc90 tx=0x7f8774001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400cea0 con 0x7f877c157830 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877400ab30 con 0x7f877c157830 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c0aabd0 con 0x7f877c157830 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87740070f0 con 0x7f877c157830 2023-10-06T01:41:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.849+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0ab190 con 0x7f877c157830 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.850+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c064ad0 con 0x7f877c157830 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.850+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87740074a0 con 0x7f877c157830 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.851+0000 7f8778ff9700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.851+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f8774084d80 con 0x7f877c157830 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.851+0000 7f8780f25700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.851+0000 7f8780f25700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 0x7f87600693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87700079f0 tx=0x7f8770008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:32.855+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8774051c60 con 0x7f877c157830 2023-10-06T01:41:33.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.079+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f877c04bb80 con 0x7f877c157830 2023-10-06T01:41:33.080 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.079+0000 7f8778ff9700 1 -- 172.21.15.98:0/1592401862 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f8774055280 con 0x7f877c157830 2023-10-06T01:41:33.080 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:33.081 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:33.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.082+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 msgr2=0x7f87600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.082+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 0x7f87600693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87700079f0 tx=0x7f8770008040 comp rx=0 tx=0).stop 2023-10-06T01:41:33.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.082+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 msgr2=0x7f877c069de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.082+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f877400cc90 tx=0x7f8774001a20 comp rx=0 tx=0).stop 2023-10-06T01:41:33.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 shutdown_connections 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f877c157830 0x7f877c069de0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877c0ac480 0x7f877c0ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 --2- 172.21.15.98:0/1592401862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877c157460 0x7f877c0a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 >> 172.21.15.98:0/1592401862 conn(0x7f877c06de80 msgr2=0x7f877c1565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 shutdown_connections 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.083+0000 7f8783189700 1 -- 172.21.15.98:0/1592401862 wait complete. 2023-10-06T01:41:33.084 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 29 2023-10-06T01:41:33.097 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:41:33.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.211+0000 7f5af9c7a700 1 Processor -- start 2023-10-06T01:41:33.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.211+0000 7f5af9c7a700 1 -- start start 2023-10-06T01:41:33.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af9c7a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af41595a0 0x7f5af40aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af9c7a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5af40b07a0 0x7f5af40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af9c7a700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40b0ca0 con 0x7f5af40b49e0 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af9c7a700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40b0f80 con 0x7f5af40b07a0 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af9c7a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40b1260 con 0x7f5af41595a0 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af3fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5af40b07a0 0x7f5af40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af2ffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5af40b49e0 0x7f5af40aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47870/0 (socket says 172.21.15.98:47870) 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af2ffd700 1 -- 172.21.15.98:0/451160645 learned_addr learned my addr 172.21.15.98:0/451160645 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af37fe700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af41595a0 0x7f5af40aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af3fff700 1 -- 172.21.15.98:0/451160645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af41595a0 msgr2=0x7f5af40aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af3fff700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af41595a0 0x7f5af40aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af3fff700 1 -- 172.21.15.98:0/451160645 >> v1:172.21.15.98:6789/0 conn(0x7f5af40b49e0 legacy=0x7f5af40aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af3fff700 1 -- 172.21.15.98:0/451160645 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40b1540 con 0x7f5af40b07a0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.212+0000 7f5af27fc700 1 -- 172.21.15.98:0/451160645 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393597206 0 0) 0x7f5af40b0ca0 con 0x7f5af40b49e0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af3fff700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40b1c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e60 tx=0x7f5aec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcf4602eec095d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af27fc700 1 -- 172.21.15.98:0/451160645 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00e070 con 0x7f5af40b07a0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af27fc700 1 -- 172.21.15.98:0/451160645 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00bc30 con 0x7f5af40b07a0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af27fc700 1 -- 172.21.15.98:0/451160645 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec012430 con 0x7f5af40b07a0 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 -- 172.21.15.98:0/451160645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 msgr2=0x7f5af40b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40b1c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e60 tx=0x7f5aec009780 comp rx=0 tx=0).stop 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 -- 172.21.15.98:0/451160645 shutdown_connections 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40b1c00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/451160645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af41595a0 0x7f5af40aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 -- 172.21.15.98:0/451160645 >> 172.21.15.98:0/451160645 conn(0x7f5af4099830 msgr2=0x7f5af406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.216 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 -- 172.21.15.98:0/451160645 shutdown_connections 2023-10-06T01:41:33.217 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.213+0000 7f5af9c7a700 1 -- 172.21.15.98:0/451160645 wait complete. 2023-10-06T01:41:33.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 Processor -- start 2023-10-06T01:41:33.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 -- start start 2023-10-06T01:41:33.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.219 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 0x7f5af409a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af41595a0 0x7f5af41698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af409b4f0 con 0x7f5af40b07a0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af409a8e0 con 0x7f5af41595a0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af9c7a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af409abc0 con 0x7f5af40b49e0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41234/0 (socket says 172.21.15.98:41234) 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.215+0000 7f5af37fe700 1 -- 172.21.15.98:0/859439969 learned_addr learned my addr 172.21.15.98:0/859439969 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af2ffd700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 0x7f5af409a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af3fff700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af41595a0 0x7f5af41698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 -- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 msgr2=0x7f5af409a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.220 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 0x7f5af409a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 -- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af41595a0 msgr2=0x7f5af41698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af41595a0 0x7f5af41698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 -- 172.21.15.98:0/859439969 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009a20 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af2ffd700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 0x7f5af409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af37fe700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c880 tx=0x7f5ae4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011600 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4011780 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af409aea0 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011b40 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.216+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af409a5d0 con 0x7f5af40b07a0 2023-10-06T01:41:33.221 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.217+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4064ad0 con 0x7f5af40b07a0 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.217+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ae4018360 con 0x7f5af40b07a0 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.218+0000 7f5adb7fe700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 0x7f5ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.218+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f5ae4084640 con 0x7f5af40b07a0 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.218+0000 7f5af2ffd700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 0x7f5ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.218+0000 7f5af2ffd700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 0x7f5ae00693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b4df0 tx=0x7f5ae8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.222 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.222+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ae4055000 con 0x7f5af40b07a0 2023-10-06T01:41:33.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.447+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af40b1820 con 0x7f5af40b07a0 2023-10-06T01:41:33.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.447+0000 7f5adb7fe700 1 -- 172.21.15.98:0/859439969 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f5ae4054b90 con 0x7f5af40b07a0 2023-10-06T01:41:33.448 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:33.449 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T01:41:04.761696+0000","modified":"2023-10-06T01:41:19.034849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5409":{"gid":5409,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1433484145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1433484145},{"type":"v1","addr":"172.21.15.98:6837","nonce":1433484145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":8,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 msgr2=0x7f5ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 0x7f5ae00693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5af40b4df0 tx=0x7f5ae8009040 comp rx=0 tx=0).stop 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 msgr2=0x7f5af40aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c880 tx=0x7f5ae4008040 comp rx=0 tx=0).stop 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 shutdown_connections 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f5ae0066f30 0x7f5ae00693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af41595a0 0x7f5af41698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af40b07a0 0x7f5af40aab90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 --2- 172.21.15.98:0/859439969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5af40b49e0 0x7f5af409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 >> 172.21.15.98:0/859439969 conn(0x7f5af4099830 msgr2=0x7f5af4158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 shutdown_connections 2023-10-06T01:41:33.452 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.450+0000 7f5af9c7a700 1 -- 172.21.15.98:0/859439969 wait complete. 2023-10-06T01:41:33.452 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 29 2023-10-06T01:41:33.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:33.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.576+0000 7fc45115a700 1 Processor -- start 2023-10-06T01:41:33.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.576+0000 7fc45115a700 1 -- start start 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc45115a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc44c06e690 0x7fc44c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc45115a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc45115a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c06f040 con 0x7fc44c156540 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc45115a700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c06f470 con 0x7fc44c06e690 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc45115a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0acd60 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44259c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc44c06e690 0x7fc44c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44259c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc44c06e690 0x7fc44c06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:54420/0 (socket says 172.21.15.98:54420) 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44ad9d700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc44c156540 0x7fc44c0c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:41328/0 (socket says 172.21.15.98:41328) 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:44622/0 (socket says 172.21.15.98:44622) 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.577+0000 7fc44ad9d700 1 -- 172.21.15.98:0/3895240821 learned_addr learned my addr 172.21.15.98:0/3895240821 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44259c700 1 -- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 msgr2=0x7fc44c0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44259c700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44259c700 1 -- 172.21.15.98:0/3895240821 >> v1:172.21.15.184:6789/0 conn(0x7fc44c156540 legacy=0x7fc44c0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44259c700 1 -- 172.21.15.98:0/3895240821 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c156910 con 0x7fc44c06e690 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44b59e700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:33.580 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44259c700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c06eb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098e0 tx=0x7fc4340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb0a536259c6c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44a59c700 1 -- 172.21.15.98:0/3895240821 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400b640 con 0x7fc44c06e690 2023-10-06T01:41:33.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44a59c700 1 -- 172.21.15.98:0/3895240821 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43400b7a0 con 0x7fc44c06e690 2023-10-06T01:41:33.581 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.578+0000 7fc44a59c700 1 -- 172.21.15.98:0/3895240821 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400bb40 con 0x7fc44c06e690 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 -- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 msgr2=0x7fc44c06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c06eb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098e0 tx=0x7fc4340091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 -- 172.21.15.98:0/3895240821 shutdown_connections 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c0ad1f0 0x7fc44c0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 --2- 172.21.15.98:0/3895240821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c06eb00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.584 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 -- 172.21.15.98:0/3895240821 >> 172.21.15.98:0/3895240821 conn(0x7fc44c0680c0 msgr2=0x7fc44c0cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 -- 172.21.15.98:0/3895240821 shutdown_connections 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.579+0000 7fc45115a700 1 -- 172.21.15.98:0/3895240821 wait complete. 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.580+0000 7fc45115a700 1 Processor -- start 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.580+0000 7fc45115a700 1 -- start start 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c156540 0x7fc44c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c154f70 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1546f0 con 0x7fc44c06e690 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc45115a700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c0520 con 0x7fc44c156540 2023-10-06T01:41:33.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41248/0 (socket says 172.21.15.98:41248) 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 -- 172.21.15.98:0/2769638653 learned_addr learned my addr 172.21.15.98:0/2769638653 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44ad9d700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44b59e700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c156540 0x7fc44c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 -- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c156540 msgr2=0x7fc44c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c156540 0x7fc44c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 -- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 msgr2=0x7fc44c154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 -- 172.21.15.98:0/2769638653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009a20 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44ad9d700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:33.586 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc44259c700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc434009f80 tx=0x7fc434017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.581+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43401bd90 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.582+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434011420 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.582+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc434009420 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.582+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340117e0 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.582+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c069d40 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.582+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c0c02f0 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.583+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc434011b50 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.583+0000 7fc4437fe700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 0x7fc42c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.583+0000 7fc44ad9d700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 0x7fc42c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.583+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6537+0+0 (secure 0 0 0) 0x7fc434085ed0 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.584+0000 7fc44ad9d700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 0x7fc42c0693b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc43c001820 tx=0x7fc43c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.588 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.588+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc434052eb0 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.771+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc44c0c0660 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.772 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.772+0000 7fc4437fe700 1 -- 172.21.15.98:0/2769638653 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17408 (secure 0 0 0) 0x7fc434015090 con 0x7fc44c0ad1f0 2023-10-06T01:41:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:33.773 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":46,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:33.079828+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T01:40:40.800463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":8,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T01:41:03.494206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:33.774 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 msgr2=0x7fc42c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 0x7fc42c0693b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc43c001820 tx=0x7fc43c009040 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 msgr2=0x7fc44c1527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc434009f80 tx=0x7fc434017000 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 shutdown_connections 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fc42c066f00 0x7fc42c0693b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44c156540 0x7fc44c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc44c06e690 0x7fc44c154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 --2- 172.21.15.98:0/2769638653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc44c0ad1f0 0x7fc44c1527c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 >> 172.21.15.98:0/2769638653 conn(0x7fc44c0680c0 msgr2=0x7fc44c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.775 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 shutdown_connections 2023-10-06T01:41:33.776 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.774+0000 7fc45115a700 1 -- 172.21.15.98:0/2769638653 wait complete. 2023-10-06T01:41:33.787 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail recovery_fs 2023-10-06T01:41:33.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.902+0000 7f3335a04700 1 Processor -- start 2023-10-06T01:41:33.903 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.902+0000 7f3335a04700 1 -- start start 2023-10-06T01:41:33.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f3335a04700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f33301594d0 0x7f33300a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f3335a04700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f333006af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.910 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f3335a04700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333006aae0 con 0x7f333006b4b0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f3335a04700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300aa030 con 0x7f33301594d0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f3335a04700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300aa2d0 con 0x7f33300af600 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f332f7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f333006b4b0 0x7f33300675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47874/0 (socket says 172.21.15.98:47874) 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.903+0000 7f332f7fe700 1 -- 172.21.15.98:0/3643488754 learned_addr learned my addr 172.21.15.98:0/3643488754 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332e7fc700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f333006af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> v2:172.21.15.184:3300/0 conn(0x7f33301594d0 0x7f33300a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100748290 0 0) 0x7f333006aae0 con 0x7f333006b4b0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3318003570 con 0x7f333006b4b0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766866142 0 0) 0x7f3318003570 con 0x7f333006b4b0 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 msgr2=0x7f333006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f333006af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 >> v2:172.21.15.184:3300/0 conn(0x7f33301594d0 msgr2=0x7f33300a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 --2- 172.21.15.98:0/3643488754 >> v2:172.21.15.184:3300/0 conn(0x7f33301594d0 0x7f33300a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300aa5b0 con 0x7f333006b4b0 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f3328001470 con 0x7f333006b4b0 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 >> v1:172.21.15.98:6789/0 conn(0x7f333006b4b0 legacy=0x7f33300675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f3318003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f3318004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3318004d50 0x7f33180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300aa030 con 0x7f3318003a60 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300aa2d0 con 0x7f3318004d50 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331800ba60 con 0x7f33300af600 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> v2:172.21.15.184:3300/0 conn(0x7f33301594d0 0x7f33300a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:33.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f3318003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332e7fc700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f3318004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332f7fe700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3318004d50 0x7f33180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 -- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 msgr2=0x7f3318004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33300af600 0x7f3318004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 -- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3318004d50 msgr2=0x7f33180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3318004d50 0x7f33180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.904+0000 7f332effd700 1 -- 172.21.15.98:0/3643488754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318003570 con 0x7f3318003a60 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f332effd700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f3318003e70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f332000e3c0 tx=0x7f332000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84edc53a18e7d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000ec10 con 0x7f3318003a60 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332000f040 con 0x7f3318003a60 2023-10-06T01:41:33.913 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f332dffb700 1 -- 172.21.15.98:0/3643488754 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320014480 con 0x7f3318003a60 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 -- 172.21.15.98:0/3643488754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 msgr2=0x7f3318003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 --2- 172.21.15.98:0/3643488754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f3318003e70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f332000e3c0 tx=0x7f332000b040 comp rx=0 tx=0).stop 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f332f7fe700 1 -- 172.21.15.98:0/3643488754 reap_dead start 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 -- 172.21.15.98:0/3643488754 shutdown_connections 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 -- 172.21.15.98:0/3643488754 >> 172.21.15.98:0/3643488754 conn(0x7f33300691d0 msgr2=0x7f33300aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 -- 172.21.15.98:0/3643488754 shutdown_connections 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.905+0000 7f3335a04700 1 -- 172.21.15.98:0/3643488754 wait complete. 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.906+0000 7f3335a04700 1 Processor -- start 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 -- start start 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3330153e00 0x7f3330169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.914 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f333016a0b0 0x7f333016c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33301635d0 con 0x7f3318003a60 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3330154250 con 0x7f333016a0b0 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f3335a04700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333016a530 con 0x7f3330153e00 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f332effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f332effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41256/0 (socket says 172.21.15.98:41256) 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.907+0000 7f332effd700 1 -- 172.21.15.98:0/1889394061 learned_addr learned my addr 172.21.15.98:0/1889394061 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332e7fc700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3330153e00 0x7f3330169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332f7fe700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f333016a0b0 0x7f333016c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 -- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3330153e00 msgr2=0x7f3330169b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3330153e00 0x7f3330169b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.915 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 -- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f333016a0b0 msgr2=0x7f333016c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f333016a0b0 0x7f333016c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 -- 172.21.15.98:0/1889394061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300afe40 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f332effd700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3320009de0 tx=0x7f3320019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332001f830 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332001f9b0 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333016d510 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332001fd70 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.908+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33300b03c0 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.909+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333005f270 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.909+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f332001d670 con 0x7f3318003a60 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.909+0000 7f3316ffd700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 0x7f331c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:33.916 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.909+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f3320087190 con 0x7f3318003a60 2023-10-06T01:41:33.917 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.910+0000 7f332e7fc700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 0x7f331c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:33.917 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.910+0000 7f332e7fc700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 0x7f331c069410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f33240093f0 tx=0x7f3324008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:33.917 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:33.914+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3320054020 con 0x7f3318003a60 2023-10-06T01:41:34.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.145+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "recovery_fs"} v 0) v1 -- 0x7f333006a490 con 0x7f3318003a60 2023-10-06T01:41:34.146 INFO:tasks.ceph.mon.a.smithi098.stderr:2023-10-06T01:41:34.146+0000 7f8a6565e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T01:41:34.150 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.150+0000 7f3316ffd700 1 -- 172.21.15.98:0/1889394061 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "recovery_fs"}]=0 recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 171+0+0 (secure 0 0 0) 0x7f3320057640 con 0x7f3318003a60 2023-10-06T01:41:34.151 INFO:tasks.ceph.mds.a.smithi098.stderr: -3639> 2023-10-06T01:41:16.030+0000 7fb8b85ba700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T01:41:34.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.152+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 msgr2=0x7f331c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.152+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 0x7f331c069410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f33240093f0 tx=0x7f3324008040 comp rx=0 tx=0).stop 2023-10-06T01:41:34.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.152+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 msgr2=0x7f33301538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.152+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3320009de0 tx=0x7f3320019000 comp rx=0 tx=0).stop 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 shutdown_connections 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f331c066f60 0x7f331c069410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f333016a0b0 0x7f333016c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3318003a60 0x7f33301538c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 --2- 172.21.15.98:0/1889394061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3330153e00 0x7f3330169b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 >> 172.21.15.98:0/1889394061 conn(0x7f33300691d0 msgr2=0x7f33300adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 shutdown_connections 2023-10-06T01:41:34.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.153+0000 7f3335a04700 1 -- 172.21.15.98:0/1889394061 wait complete. 2023-10-06T01:41:34.155 INFO:tasks.ceph.mds.a.smithi098.stderr: -2043> 2023-10-06T01:41:21.072+0000 7fb8b85ba700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-06T01:41:34.155 INFO:tasks.ceph.mds.a.smithi098.stderr: -1854> 2023-10-06T01:41:21.073+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56522a922b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T01:41:34.155 INFO:tasks.ceph.mds.a.smithi098.stderr: -1837> 2023-10-06T01:41:21.074+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56522a99b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/sixmegs v10>,<0x1/subdir v12>]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T01:41:34.155 INFO:teuthology.orchestra.run.smithi098.stderr:recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T01:41:34.156 INFO:tasks.ceph.mds.a.smithi098.stderr: -1597> 2023-10-06T01:41:21.077+0000 7fb8b85ba700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 ap=5 snaprealm=0x565229b64c00 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) | dirtyscattered=0 lock=3 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x56522a922580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T01:41:34.156 INFO:tasks.ceph.mds.a.smithi098.stderr: -1304> 2023-10-06T01:41:22.059+0000 7fb8bbdc1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T01:41:34.167 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm recovery_fs --yes-i-really-mean-it 2023-10-06T01:41:34.178 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2023-10-06T01:41:34.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.282+0000 7f0892a99700 1 Processor -- start 2023-10-06T01:41:34.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0892a99700 1 -- start start 2023-10-06T01:41:34.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0892a99700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f088c0cbb50 0x7f088c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:34.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0892a99700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0b2e60 con 0x7f088c0b2a90 2023-10-06T01:41:34.285 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0892a99700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0cf5d0 con 0x7f088c0c6610 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0892a99700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0cf8b0 con 0x7f088c0cbb50 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0890835700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f088c0b2a90 0x7f088c0cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47888/0 (socket says 172.21.15.98:47888) 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.283+0000 7f0890835700 1 -- 172.21.15.98:0/1695481824 learned_addr learned my addr 172.21.15.98:0/1695481824 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088bfff700 1 --2- 172.21.15.98:0/1695481824 >> v2:172.21.15.98:3300/0 conn(0x7f088c0cbb50 0x7f088c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088b7fe700 1 -- 172.21.15.98:0/1695481824 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482123037 0 0) 0x7f088c0b2e60 con 0x7f088c0b2a90 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088b7fe700 1 -- 172.21.15.98:0/1695481824 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c003570 con 0x7f088c0b2a90 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088bfff700 1 -- 172.21.15.98:0/1695481824 >> v1:172.21.15.184:6789/0 conn(0x7f088c0c6610 legacy=0x7f088c0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088bfff700 1 -- 172.21.15.98:0/1695481824 >> v1:172.21.15.98:6789/0 conn(0x7f088c0b2a90 legacy=0x7f088c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088bfff700 1 -- 172.21.15.98:0/1695481824 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0cfb90 con 0x7f088c0cbb50 2023-10-06T01:41:34.286 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088bfff700 1 --2- 172.21.15.98:0/1695481824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c0ce850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f08780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f3aa42689193ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088b7fe700 1 -- 172.21.15.98:0/1695481824 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800b640 con 0x7f088c0cbb50 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088b7fe700 1 -- 172.21.15.98:0/1695481824 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800b7a0 con 0x7f088c0cbb50 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.284+0000 7f088b7fe700 1 -- 172.21.15.98:0/1695481824 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800bb40 con 0x7f088c0cbb50 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 -- 172.21.15.98:0/1695481824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 msgr2=0x7f088c0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 --2- 172.21.15.98:0/1695481824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c0ce850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f08780091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 -- 172.21.15.98:0/1695481824 shutdown_connections 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 --2- 172.21.15.98:0/1695481824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c0ce850 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 -- 172.21.15.98:0/1695481824 >> 172.21.15.98:0/1695481824 conn(0x7f088c099740 msgr2=0x7f088c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 -- 172.21.15.98:0/1695481824 shutdown_connections 2023-10-06T01:41:34.287 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.285+0000 7f0892a99700 1 -- 172.21.15.98:0/1695481824 wait complete. 2023-10-06T01:41:34.289 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.286+0000 7f0892a99700 1 Processor -- start 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.286+0000 7f0892a99700 1 -- start start 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088c0b2a90 0x7f088c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f088c0c6610 0x7f088c09ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c09cdb0 con 0x7f088c0cbb50 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c09b890 con 0x7f088c0c6610 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c09d630 con 0x7f088c0b2a90 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41292/0 (socket says 172.21.15.98:41292) 2023-10-06T01:41:34.290 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 -- 172.21.15.98:0/2408007802 learned_addr learned my addr 172.21.15.98:0/2408007802 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0890835700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088c0b2a90 0x7f088c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 -- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088c0b2a90 msgr2=0x7f088c16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f088bfff700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f088c0c6610 0x7f088c09ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088c0b2a90 0x7f088c16a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 -- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f088c0c6610 msgr2=0x7f088c09ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f088c0c6610 0x7f088c09ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 -- 172.21.15.98:0/2408007802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878009420 con 0x7f088c0cbb50 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0891036700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f088400c8e0 tx=0x7f0884001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400ce80 con 0x7f088c0cbb50 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088400a7e0 con 0x7f088c0cbb50 2023-10-06T01:41:34.291 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400ab80 con 0x7f088c0cbb50 2023-10-06T01:41:34.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.287+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c09b5b0 con 0x7f088c0cbb50 2023-10-06T01:41:34.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.288+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c09d2c0 con 0x7f088c0cbb50 2023-10-06T01:41:34.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.288+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c09d7a0 con 0x7f088c0cbb50 2023-10-06T01:41:34.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.289+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0884012420 con 0x7f088c0cbb50 2023-10-06T01:41:34.292 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.289+0000 7f0888ff9700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 0x7f08700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:34.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.289+0000 7f0890835700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 0x7f08700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:34.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.289+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f0884084b40 con 0x7f088c0cbb50 2023-10-06T01:41:34.293 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.290+0000 7f0890835700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 0x7f08700693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0880007fd0 tx=0x7f0880009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:34.295 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.294+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0884051a20 con 0x7f088c0cbb50 2023-10-06T01:41:34.523 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:34.523+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f088c09b320 con 0x7f088c0cbb50 2023-10-06T01:41:35.151 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.150+0000 7f0888ff9700 1 -- 172.21.15.98:0/2408007802 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 109+0+0 (secure 0 0 0) 0x7f0884055040 con 0x7f088c0cbb50 2023-10-06T01:41:35.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 msgr2=0x7f08700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 0x7f08700693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0880007fd0 tx=0x7f0880009040 comp rx=0 tx=0).stop 2023-10-06T01:41:35.153 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 msgr2=0x7f088c155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f088400c8e0 tx=0x7f0884001710 comp rx=0 tx=0).stop 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 shutdown_connections 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f0870066f30 0x7f08700693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088c0cbb50 0x7f088c155c20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088c0b2a90 0x7f088c16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 --2- 172.21.15.98:0/2408007802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f088c0c6610 0x7f088c09ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.152+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 >> 172.21.15.98:0/2408007802 conn(0x7f088c099740 msgr2=0x7f088c0d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.153+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 shutdown_connections 2023-10-06T01:41:35.154 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.153+0000 7f0892a99700 1 -- 172.21.15.98:0/2408007802 wait complete. 2023-10-06T01:41:35.166 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm recovery_fs_metadata recovery_fs_metadata --yes-i-really-really-mean-it 2023-10-06T01:41:35.326 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 Processor -- start 2023-10-06T01:41:35.326 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 -- start start 2023-10-06T01:41:35.329 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8c1c15a780 0x7f8c1c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.329 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0b2d20 con 0x7f8c1c06ed20 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0b0450 con 0x7f8c1c0b23a0 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c22100700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c06fb80 con 0x7f8c1c15a780 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c1b7fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8c1c06ed20 0x7f8c1c06f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47904/0 (socket says 172.21.15.98:47904) 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.326+0000 7f8c1b7fe700 1 -- 172.21.15.98:0/715239272 learned_addr learned my addr 172.21.15.98:0/715239272 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/715239272 >> v2:172.21.15.184:3300/0 conn(0x7f8c1c15a780 0x7f8c1c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725435195 0 0) 0x7f8c1c0b2d20 con 0x7f8c1c06ed20 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c04003570 con 0x7f8c1c06ed20 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676135589 0 0) 0x7f8c04003570 con 0x7f8c1c06ed20 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 >> v2:172.21.15.184:3300/0 conn(0x7f8c1c15a780 msgr2=0x7f8c1c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 --2- 172.21.15.98:0/715239272 >> v2:172.21.15.184:3300/0 conn(0x7f8c1c15a780 0x7f8c1c0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.330 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 >> v1:172.21.15.184:6789/0 conn(0x7f8c1c0b23a0 legacy=0x7f8c1c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0b2a40 con 0x7f8c1c06ed20 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f8c0c0017f0 con 0x7f8c1c06ed20 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 >> v1:172.21.15.98:6789/0 conn(0x7f8c1c06ed20 legacy=0x7f8c1c06f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04003a60 0x7f8c04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 0x7f8c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c040051c0 0x7f8c0400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0400c050 con 0x7f8c04004810 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0400c190 con 0x7f8c040051c0 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0400c350 con 0x7f8c04003a60 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 0x7f8c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.331 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c040051c0 0x7f8c0400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04003a60 0x7f8c04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 -- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04003a60 msgr2=0x7f8c04003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04003a60 0x7f8c04003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 -- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c040051c0 msgr2=0x7f8c0400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c040051c0 0x7f8c0400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 -- 172.21.15.98:0/715239272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04003570 con 0x7f8c04004810 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04003a60 0x7f8c04003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c040051c0 0x7f8c0400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1bfff700 1 -- 172.21.15.98:0/715239272 reap_dead start 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.327+0000 7f8c1affd700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 0x7f8c04004c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8c100019a0 tx=0x7f8c10007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e35c9f0c217e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000c5d0 con 0x7f8c04004810 2023-10-06T01:41:35.332 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1000c730 con 0x7f8c04004810 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c1a7fc700 1 -- 172.21.15.98:0/715239272 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000cad0 con 0x7f8c04004810 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 -- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 msgr2=0x7f8c04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 0x7f8c04004c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8c100019a0 tx=0x7f8c10007d10 comp rx=0 tx=0).stop 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 -- 172.21.15.98:0/715239272 shutdown_connections 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 --2- 172.21.15.98:0/715239272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c04004810 0x7f8c04004c80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 -- 172.21.15.98:0/715239272 >> 172.21.15.98:0/715239272 conn(0x7f8c1c068330 msgr2=0x7f8c1c068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 -- 172.21.15.98:0/715239272 shutdown_connections 2023-10-06T01:41:35.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.328+0000 7f8c22100700 1 -- 172.21.15.98:0/715239272 wait complete. 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.329+0000 7f8c22100700 1 Processor -- start 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.329+0000 7f8c22100700 1 -- start start 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 0x7f8c1c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 0x7f8c1c0a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c169ab0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0a72e0 con 0x7f8c1c0a8e70 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c22100700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0a7510 con 0x7f8c04004810 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41320/0 (socket says 172.21.15.98:41320) 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 -- 172.21.15.98:0/21577464 learned_addr learned my addr 172.21.15.98:0/21577464 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 0x7f8c1c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1affd700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 0x7f8c1c0a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 -- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 msgr2=0x7f8c1c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 0x7f8c1c0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 -- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 msgr2=0x7f8c1c0a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 0x7f8c1c0a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.330+0000 7f8c1bfff700 1 -- 172.21.15.98:0/21577464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0a77f0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 0x7f8c1c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c1affd700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 0x7f8c1c0a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c1bfff700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c069370 tx=0x7f8c14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14015670 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c140157f0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0a7ad0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14015bb0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.331+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c0a92e0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.332+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0a7cc0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.332+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c14020460 con 0x7f8c1c16ac60 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.333+0000 7f8c037fe700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 0x7f8c080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:35.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.333+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f8c14084db0 con 0x7f8c1c16ac60 2023-10-06T01:41:35.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.333+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 0x7f8c080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:35.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.333+0000 7f8c1b7fe700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 0x7f8c080693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0012e0 tx=0x7f8c0c001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:35.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.337+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c14051c90 con 0x7f8c1c16ac60 2023-10-06T01:41:35.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:35.518+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c1c064ad0 con 0x7f8c1c16ac60 2023-10-06T01:41:36.155 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.154+0000 7f8c037fe700 1 -- 172.21.15.98:0/21577464 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'recovery_fs_metadata' removed v49) v1 ==== 196+0+0 (secure 0 0 0) 0x7f8c140552b0 con 0x7f8c1c16ac60 2023-10-06T01:41:36.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 msgr2=0x7f8c080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:36.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 0x7f8c080693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0012e0 tx=0x7f8c0c001520 comp rx=0 tx=0).stop 2023-10-06T01:41:36.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 msgr2=0x7f8c1c16b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:36.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c069370 tx=0x7f8c14008040 comp rx=0 tx=0).stop 2023-10-06T01:41:36.158 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 shutdown_connections 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f8c08066f30 0x7f8c080693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c1c16ac60 0x7f8c1c16b0b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c04004810 0x7f8c1c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 --2- 172.21.15.98:0/21577464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c0a8e70 0x7f8c1c0a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 >> 172.21.15.98:0/21577464 conn(0x7f8c1c068330 msgr2=0x7f8c1c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 shutdown_connections 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.157+0000 7f8c22100700 1 -- 172.21.15.98:0/21577464 wait complete. 2023-10-06T01:41:36.159 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'recovery_fs_metadata' removed 2023-10-06T01:41:36.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T01:41:36.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.296+0000 7f4ecc8da700 1 Processor -- start 2023-10-06T01:41:36.297 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.296+0000 7f4ecc8da700 1 -- start start 2023-10-06T01:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ecc8da700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806ed20 0x7f4ec80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ecc8da700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ec80aee20 0x7f4ec80af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ecc8da700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec80b2d20 con 0x7f4ec806f0f0 2023-10-06T01:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ecc8da700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec806fb80 con 0x7f4ec80aee20 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ecc8da700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80b0450 con 0x7f4ec806ed20 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ec80aee20 0x7f4ec80af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ec80aee20 0x7f4ec80af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41328/0 (socket says 172.21.15.98:41328) 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/2810967985 learned_addr learned my addr 172.21.15.98:0/2810967985 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/2810967985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806ed20 msgr2=0x7f4ec80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 --2- 172.21.15.98:0/2810967985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806ed20 0x7f4ec80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/2810967985 >> v1:172.21.15.184:6789/0 conn(0x7f4ec806f0f0 legacy=0x7f4ec80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/2810967985 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80af7c0 con 0x7f4ec80aee20 2023-10-06T01:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec6d9d700 1 --2- 172.21.15.98:0/2810967985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80af280 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009f20 tx=0x7f4ec0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33e0c8a505de8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec5d9b700 1 -- 172.21.15.98:0/2810967985 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000e070 con 0x7f4ec80aee20 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec5d9b700 1 -- 172.21.15.98:0/2810967985 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000ba40 con 0x7f4ec80aee20 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.297+0000 7f4ec5d9b700 1 -- 172.21.15.98:0/2810967985 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00092e0 con 0x7f4ec80aee20 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 -- 172.21.15.98:0/2810967985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 msgr2=0x7f4ec80af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/2810967985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80af280 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009f20 tx=0x7f4ec0009840 comp rx=0 tx=0).stop 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 -- 172.21.15.98:0/2810967985 shutdown_connections 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/2810967985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80af280 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/2810967985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806ed20 0x7f4ec80b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 -- 172.21.15.98:0/2810967985 >> 172.21.15.98:0/2810967985 conn(0x7f4ec8068330 msgr2=0x7f4ec8068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 -- 172.21.15.98:0/2810967985 shutdown_connections 2023-10-06T01:41:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.298+0000 7f4ecc8da700 1 -- 172.21.15.98:0/2810967985 wait complete. 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.299+0000 7f4ecc8da700 1 Processor -- start 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.299+0000 7f4ecc8da700 1 -- start start 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 0x7f4ec8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 0x7f4ec81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80c74e0 con 0x7f4ec80aee20 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8169ec0 con 0x7f4ec806ed20 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ecc8da700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec816a1a0 con 0x7f4ec806f0f0 2023-10-06T01:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41334/0 (socket says 172.21.15.98:41334) 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/971127318 learned_addr learned my addr 172.21.15.98:0/971127318 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ebdd9b700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 0x7f4ec81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec659c700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 0x7f4ec8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 msgr2=0x7f4ec81487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 0x7f4ec81487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 msgr2=0x7f4ec8157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 0x7f4ec8157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 -- 172.21.15.98:0/971127318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009ae0 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ebdd9b700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 0x7f4ec81487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec659c700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 0x7f4ec8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.300+0000 7f4ec6d9d700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80893c0 tx=0x7f4ec0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.301+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000beb0 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.301+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0012e60 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.301+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec816a480 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.301+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001e6a0 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.301+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec8157c70 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.302+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec8070060 con 0x7f4ec80aee20 2023-10-06T01:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.302+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ec001ead0 con 0x7f4ec80aee20 2023-10-06T01:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.302+0000 7f4ebeffd700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.302+0000 7f4ec659c700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.302+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6175+0+0 (secure 0 0 0) 0x7f4ec0086430 con 0x7f4ec80aee20 2023-10-06T01:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.303+0000 7f4ec659c700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006fd0 tx=0x7f4eb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.307+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec00534d0 con 0x7f4ec80aee20 2023-10-06T01:41:36.488 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:36.487+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4ec816a920 con 0x7f4ec80aee20 2023-10-06T01:41:37.161 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.160+0000 7f4ebeffd700 1 -- 172.21.15.98:0/971127318 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4ec0056af0 con 0x7f4ec80aee20 2023-10-06T01:41:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.163+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 msgr2=0x7f4ea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.164 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.163+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006fd0 tx=0x7f4eb8008040 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.163+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 msgr2=0x7f4ec80c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.163+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80893c0 tx=0x7f4ec0017000 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 shutdown_connections 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ec80aee20 0x7f4ec80c8cc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec806ed20 0x7f4ec8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 --2- 172.21.15.98:0/971127318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec806f0f0 0x7f4ec81487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 >> 172.21.15.98:0/971127318 conn(0x7f4ec8068330 msgr2=0x7f4ec8156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 shutdown_connections 2023-10-06T01:41:37.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.164+0000 7f4ecc8da700 1 -- 172.21.15.98:0/971127318 wait complete. 2023-10-06T01:41:37.166 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2023-10-06T01:41:37.178 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2023-10-06T01:41:37.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 Processor -- start 2023-10-06T01:41:37.296 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 -- start start 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 0x7fb180157d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb180158250 0x7fb180158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800afbf0 con 0x7fb18015b980 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800afed0 con 0x7fb180158250 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.296+0000 7fb186dcf700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800b01b0 con 0x7fb18006bff0 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb180158250 0x7fb180158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb180158250 0x7fb180158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41338/0 (socket says 172.21.15.98:41338) 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 -- 172.21.15.98:0/2613076110 learned_addr learned my addr 172.21.15.98:0/2613076110 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:37.299 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb17ffff700 1 --2- 172.21.15.98:0/2613076110 >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 0x7fb180157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 -- 172.21.15.98:0/2613076110 >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 msgr2=0x7fb180157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 --2- 172.21.15.98:0/2613076110 >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 0x7fb180157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 -- 172.21.15.98:0/2613076110 >> v1:172.21.15.184:6789/0 conn(0x7fb18015b980 legacy=0x7fb18006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 -- 172.21.15.98:0/2613076110 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800b0490 con 0x7fb180158250 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb17ffff700 1 --2- 172.21.15.98:0/2613076110 >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 0x7fb180157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb18536c700 1 --2- 172.21.15.98:0/2613076110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb180158aa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb1800c1d30 tx=0x7fb178009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a78a05820ec625c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb17f7fe700 1 -- 172.21.15.98:0/2613076110 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17800b820 con 0x7fb180158250 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb17f7fe700 1 -- 172.21.15.98:0/2613076110 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17800b980 con 0x7fb180158250 2023-10-06T01:41:37.300 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.297+0000 7fb17f7fe700 1 -- 172.21.15.98:0/2613076110 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17800bd40 con 0x7fb180158250 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 -- 172.21.15.98:0/2613076110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 msgr2=0x7fb180158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 --2- 172.21.15.98:0/2613076110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb180158aa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb1800c1d30 tx=0x7fb178009250 comp rx=0 tx=0).stop 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 -- 172.21.15.98:0/2613076110 shutdown_connections 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 --2- 172.21.15.98:0/2613076110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb180158aa0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 --2- 172.21.15.98:0/2613076110 >> v2:172.21.15.184:3300/0 conn(0x7fb18006bff0 0x7fb180157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 -- 172.21.15.98:0/2613076110 >> 172.21.15.98:0/2613076110 conn(0x7fb1800ce330 msgr2=0x7fb1800ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 -- 172.21.15.98:0/2613076110 shutdown_connections 2023-10-06T01:41:37.301 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.298+0000 7fb186dcf700 1 -- 172.21.15.98:0/2613076110 wait complete. 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.299+0000 7fb186dcf700 1 Processor -- start 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.299+0000 7fb186dcf700 1 -- start start 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18006bff0 0x7fb18014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 0x7fb1800b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b2460 con 0x7fb180158250 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b38f0 con 0x7fb18015b980 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800c6d60 con 0x7fb18006bff0 2023-10-06T01:41:37.304 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41342/0 (socket says 172.21.15.98:41342) 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 -- 172.21.15.98:0/1465902948 learned_addr learned my addr 172.21.15.98:0/1465902948 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb184b6b700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18006bff0 0x7fb18014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb18536c700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 0x7fb1800b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 -- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18006bff0 msgr2=0x7fb18014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18006bff0 0x7fb18014a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 -- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 msgr2=0x7fb1800b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 0x7fb1800b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 -- 172.21.15.98:0/1465902948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780094f0 con 0x7fb180158250 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb18536c700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 0x7fb1800b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:37.305 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17ffff700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c800 tx=0x7fb174008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011600 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174011760 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1800b2180 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011b20 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.300+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1800b3ef0 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.301+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180064ad0 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.302+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1740193b0 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.302+0000 7fb17cff9700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 0x7fb16c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.302+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fb1740845e0 con 0x7fb180158250 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.302+0000 7fb184b6b700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 0x7fb16c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.306 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.302+0000 7fb184b6b700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 0x7fb16c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb170009fd0 tx=0x7fb170009600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.307 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.306+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb174051850 con 0x7fb180158250 2023-10-06T01:41:37.489 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.489+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) v1 -- 0x7fb1800af8c0 con 0x7fb180158250 2023-10-06T01:41:37.492 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.492+0000 7fb17cff9700 1 -- 172.21.15.98:0/1465902948 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v12) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb174054e70 con 0x7fb180158250 2023-10-06T01:41:37.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 msgr2=0x7fb16c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 0x7fb16c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb170009fd0 tx=0x7fb170009600 comp rx=0 tx=0).stop 2023-10-06T01:41:37.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 msgr2=0x7fb1800b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.495 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c800 tx=0x7fb174008040 comp rx=0 tx=0).stop 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 shutdown_connections 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb16c066f30 0x7fb16c0693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb18015b980 0x7fb1800b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb18006bff0 0x7fb18014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 --2- 172.21.15.98:0/1465902948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb180158250 0x7fb1800b12f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 >> 172.21.15.98:0/1465902948 conn(0x7fb1800ce330 msgr2=0x7fb1800ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 shutdown_connections 2023-10-06T01:41:37.496 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.494+0000 7fb186dcf700 1 -- 172.21.15.98:0/1465902948 wait complete. 2023-10-06T01:41:37.507 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T01:41:37.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.621+0000 7fa001958700 1 Processor -- start 2023-10-06T01:41:37.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.621+0000 7fa001958700 1 -- start start 2023-10-06T01:41:37.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7fa001958700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9ffc0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.625 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7fa001958700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ffc0c6610 0x7f9ffc0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7fa001958700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ffc0b2e60 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7fa001958700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cf5d0 con 0x7f9ffc0c6610 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7fa001958700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cf8b0 con 0x7f9ffc0cbb50 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ffaffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9ffc0b2a90 0x7f9ffc0cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47908/0 (socket says 172.21.15.98:47908) 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ffaffd700 1 -- 172.21.15.98:0/487396886 learned_addr learned my addr 172.21.15.98:0/487396886 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ff27fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9ffc0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ffb7fe700 1 --2- 172.21.15.98:0/487396886 >> v2:172.21.15.184:3300/0 conn(0x7f9ffc0c6610 0x7f9ffc0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786779419 0 0) 0x7f9ffc0b2e60 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.626 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.622+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe8003570 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106161358 0 0) 0x7f9fe8003570 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 msgr2=0x7f9ffc0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9ffc0ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 >> v2:172.21.15.184:3300/0 conn(0x7f9ffc0c6610 msgr2=0x7f9ffc0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 --2- 172.21.15.98:0/487396886 >> v2:172.21.15.184:3300/0 conn(0x7f9ffc0c6610 0x7f9ffc0ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc0cfb90 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f9fec0015f0 con 0x7f9ffc0b2a90 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 >> v1:172.21.15.98:6789/0 conn(0x7f9ffc0b2a90 legacy=0x7f9ffc0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9fe8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9fe8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.627 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fe8004d50 0x7f9fe80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cf5d0 con 0x7f9fe8003a60 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cf8b0 con 0x7f9fe8004d50 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe800ba60 con 0x7f9ffc0cbb50 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9fe8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ff27fc700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9fe8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffb7fe700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fe8004d50 0x7f9fe80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 -- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 msgr2=0x7f9fe8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc0cbb50 0x7f9fe8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 -- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fe8004d50 msgr2=0x7f9fe80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fe8004d50 0x7f9fe80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 -- 172.21.15.98:0/487396886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8003570 con 0x7f9fe8003a60 2023-10-06T01:41:37.628 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffb7fe700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fe8004d50 0x7f9fe80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9fe8003e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c3c0 tx=0x7f9fec007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d08e23871efc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00cc30 con 0x7f9fe8003a60 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00cd90 con 0x7f9fe8003a60 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.623+0000 7f9ffa7fc700 1 -- 172.21.15.98:0/487396886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec007320 con 0x7f9fe8003a60 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 -- 172.21.15.98:0/487396886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 msgr2=0x7f9fe8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 --2- 172.21.15.98:0/487396886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9fe8003e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c3c0 tx=0x7f9fec007920 comp rx=0 tx=0).stop 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7f9ffb7fe700 1 -- 172.21.15.98:0/487396886 reap_dead start 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 -- 172.21.15.98:0/487396886 shutdown_connections 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 -- 172.21.15.98:0/487396886 >> 172.21.15.98:0/487396886 conn(0x7f9ffc099740 msgr2=0x7f9ffc06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 -- 172.21.15.98:0/487396886 shutdown_connections 2023-10-06T01:41:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.624+0000 7fa001958700 1 -- 172.21.15.98:0/487396886 wait complete. 2023-10-06T01:41:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.625+0000 7fa001958700 1 Processor -- start 2023-10-06T01:41:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.625+0000 7fa001958700 1 -- start start 2023-10-06T01:41:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc09a260 0x7f9ffc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 0x7f9ffc09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc09b3e0 con 0x7f9fe8003a60 2023-10-06T01:41:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc09d600 con 0x7f9ffc09cc50 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7fa001958700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc148780 con 0x7f9ffc09a260 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41358/0 (socket says 172.21.15.98:41358) 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 -- 172.21.15.98:0/3326997521 learned_addr learned my addr 172.21.15.98:0/3326997521 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ff27fc700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc09a260 0x7f9ffc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffb7fe700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 0x7f9ffc09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 -- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc09a260 msgr2=0x7f9ffc15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc09a260 0x7f9ffc15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 -- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 msgr2=0x7f9ffc09d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 0x7f9ffc09d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 -- 172.21.15.98:0/3326997521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc09a7d0 con 0x7f9fe8003a60 2023-10-06T01:41:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffb7fe700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 0x7f9ffc09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.626+0000 7f9ffaffd700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8004730 tx=0x7f9fec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.627+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00cc30 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.627+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec001160 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.627+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc09aab0 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.627+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec001de0 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.627+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc09b070 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.628+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc064ad0 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.628+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fec019030 con 0x7f9fe8003a60 2023-10-06T01:41:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.628+0000 7f9ff37fe700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 0x7f9fdc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.628+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f9fec085950 con 0x7f9fe8003a60 2023-10-06T01:41:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.628+0000 7f9ff27fc700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 0x7f9fdc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.629+0000 7f9ff27fc700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 0x7f9fdc069460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc09d870 tx=0x7f9fe40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.632+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fec052b20 con 0x7f9fe8003a60 2023-10-06T01:41:37.817 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.816+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9ffc0b61a0 con 0x7f9fe8003a60 2023-10-06T01:41:37.820 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.820+0000 7f9ff37fe700 1 -- 172.21.15.98:0/3326997521 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9fec0170a0 con 0x7f9fe8003a60 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 msgr2=0x7f9fdc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 0x7f9fdc069460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc09d870 tx=0x7f9fe40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 msgr2=0x7f9ffc148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8004730 tx=0x7f9fec015000 comp rx=0 tx=0).stop 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 shutdown_connections 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9fdc066fb0 0x7f9fdc069460 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ffc09cc50 0x7f9ffc09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.823 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe8003a60 0x7f9ffc148240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 --2- 172.21.15.98:0/3326997521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ffc09a260 0x7f9ffc15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 >> 172.21.15.98:0/3326997521 conn(0x7f9ffc099740 msgr2=0x7f9ffc069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 shutdown_connections 2023-10-06T01:41:37.824 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.822+0000 7fa001958700 1 -- 172.21.15.98:0/3326997521 wait complete. 2023-10-06T01:41:37.835 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2023-10-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.956+0000 7fa16da0f700 1 Processor -- start 2023-10-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 -- start start 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1681595a0 0x7fa1680aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680b0ca0 con 0x7fa1680b49e0 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1680b0f80 con 0x7fa1681595a0 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.957+0000 7fa16da0f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b1260 con 0x7fa1680b07a0 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1677fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1681595a0 0x7fa1680aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa15e7fc700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa1680b49e0 0x7fa1680aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:41376/0 (socket says 172.21.15.98:41376) 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1681595a0 0x7fa1680aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:54584/0 (socket says 172.21.15.98:54584) 2023-10-06T01:41:37.960 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa15e7fc700 1 -- 172.21.15.98:0/3553213385 learned_addr learned my addr 172.21.15.98:0/3553213385 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1677fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:44726/0 (socket says 172.21.15.98:44726) 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 -- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 msgr2=0x7fa1680b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 -- 172.21.15.98:0/3553213385 >> v1:172.21.15.184:6789/0 conn(0x7fa1680b49e0 legacy=0x7fa1680aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 -- 172.21.15.98:0/3553213385 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680b1540 con 0x7fa1681595a0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1667fc700 1 -- 172.21.15.98:0/3553213385 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355940250 0 0) 0x7fa1680b0ca0 con 0x7fa1680b49e0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1677fe700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa166ffd700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680aa570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa158009900 tx=0x7fa158009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0fb21c4eb06dfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1667fc700 1 -- 172.21.15.98:0/3553213385 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158005c10 con 0x7fa1681595a0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1667fc700 1 -- 172.21.15.98:0/3553213385 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158005d70 con 0x7fa1681595a0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.958+0000 7fa1667fc700 1 -- 172.21.15.98:0/3553213385 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158006260 con 0x7fa1681595a0 2023-10-06T01:41:37.961 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 -- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 msgr2=0x7fa1680aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680aa570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa158009900 tx=0x7fa158009120 comp rx=0 tx=0).stop 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 -- 172.21.15.98:0/3553213385 shutdown_connections 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 --2- 172.21.15.98:0/3553213385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680aa570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 -- 172.21.15.98:0/3553213385 >> 172.21.15.98:0/3553213385 conn(0x7fa168099830 msgr2=0x7fa16806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 -- 172.21.15.98:0/3553213385 shutdown_connections 2023-10-06T01:41:37.962 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.959+0000 7fa16da0f700 1 -- 172.21.15.98:0/3553213385 wait complete. 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.960+0000 7fa16da0f700 1 Processor -- start 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.960+0000 7fa16da0f700 1 -- start start 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168157be0 con 0x7fa1680b49e0 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168147460 con 0x7fa1681595a0 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa16da0f700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680abbf0 con 0x7fa1680b07a0 2023-10-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41362/0 (socket says 172.21.15.98:41362) 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 -- 172.21.15.98:0/716859426 learned_addr learned my addr 172.21.15.98:0/716859426 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa166ffd700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa1677fe700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 -- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 msgr2=0x7fa1680adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 -- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 msgr2=0x7fa1680ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.961+0000 7fa15e7fc700 1 -- 172.21.15.98:0/716859426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160009a20 con 0x7fa1680b49e0 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa15e7fc700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa15000c810 tx=0x7fa150007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150005c00 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150005d60 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1580093c0 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150007300 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.962+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1680ae810 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.963+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168064ad0 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.963+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa15001b3b0 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.963+0000 7fa15f7fe700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 0x7fa1480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.964+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fa150083980 con 0x7fa1680b49e0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.964+0000 7fa166ffd700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 0x7fa1480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.964+0000 7fa166ffd700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 0x7fa1480693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa158009160 tx=0x7fa158016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:37.968 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:37.968+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa150050cb0 con 0x7fa1680b49e0 2023-10-06T01:41:38.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:38.230+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) v1 -- 0x7fa1680b1820 con 0x7fa1680b49e0 2023-10-06T01:41:39.162 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.162+0000 7fa15f7fe700 1 -- 172.21.15.98:0/716859426 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v127) v1 ==== 147+0+0 (secure 0 0 0) 0x7fa1500170d0 con 0x7fa1680b49e0 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 msgr2=0x7fa1480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 0x7fa1480693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa158009160 tx=0x7fa158016000 comp rx=0 tx=0).stop 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 msgr2=0x7fa1681576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa15000c810 tx=0x7fa150007d90 comp rx=0 tx=0).stop 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 shutdown_connections 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa148066f30 0x7fa1480693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1681595a0 0x7fa1680ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1680b07a0 0x7fa1680adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:39.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 --2- 172.21.15.98:0/716859426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1680b49e0 0x7fa1681576a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:39.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 >> 172.21.15.98:0/716859426 conn(0x7fa168099830 msgr2=0x7fa168158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:39.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 shutdown_connections 2023-10-06T01:41:39.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:39.164+0000 7fa16da0f700 1 -- 172.21.15.98:0/716859426 wait complete. 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner:test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) ... ok 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner: 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner:Ran 1 test in 74.395s 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner: 2023-10-06T01:41:39.178 INFO:tasks.cephfs_test_runner:OK 2023-10-06T01:41:39.179 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T01:41:39.191 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T01:41:39.211 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T01:41:39.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.211 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:41:39.226 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T01:41:39.227 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T01:41:39.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T01:41:39.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.311 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T01:41:39.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2023-10-06T01:41:39.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.491 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2023-10-06T01:41:39.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.506 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2023-10-06T01:41:39.574 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2023-10-06T01:41:39.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:41:39.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.576 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:41:39.594 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T01:41:39.594 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T01:41:39.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T01:41:39.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.678 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T01:41:39.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-06T01:41:39.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.859 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-06T01:41:39.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T01:41:39.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-06T01:41:39.942 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-06T01:41:39.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:41:39.943 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T01:41:39.952 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T01:41:39.953 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:40.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.067+0000 7fdff0cb4700 1 Processor -- start 2023-10-06T01:41:40.068 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.067+0000 7fdff0cb4700 1 -- start start 2023-10-06T01:41:40.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdff0cb4700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfec06f310 0x7fdfec15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdff0cb4700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec06d960 con 0x7fdfec140b80 2023-10-06T01:41:40.069 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdff0cb4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0af3d0 con 0x7fdfec1407b0 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdff0cb4700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0c6540 con 0x7fdfec06f310 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfebfff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfec06f310 0x7fdfec15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfeaffd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdfec140b80 0x7fdfec0aecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47924/0 (socket says 172.21.15.98:47924) 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfeaffd700 1 -- 172.21.15.98:0/4004299837 learned_addr learned my addr 172.21.15.98:0/4004299837 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfebfff700 1 -- 172.21.15.98:0/4004299837 >> v1:172.21.15.184:6789/0 conn(0x7fdfec1407b0 legacy=0x7fdfec06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfebfff700 1 -- 172.21.15.98:0/4004299837 >> v1:172.21.15.98:6789/0 conn(0x7fdfec140b80 legacy=0x7fdfec0aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfebfff700 1 -- 172.21.15.98:0/4004299837 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0c6820 con 0x7fdfec06f310 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.068+0000 7fdfea7fc700 1 -- 172.21.15.98:0/4004299837 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688714083 0 0) 0x7fdfe0001110 con 0x7fdfec140b80 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdfebfff700 1 --2- 172.21.15.98:0/4004299837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec06f310 0x7fdfec15adf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2901359ff8cb0d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdfea7fc700 1 -- 172.21.15.98:0/4004299837 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4009e80 con 0x7fdfec06f310 2023-10-06T01:41:40.070 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdfea7fc700 1 -- 172.21.15.98:0/4004299837 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400b820 con 0x7fdfec06f310 2023-10-06T01:41:40.071 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdfea7fc700 1 -- 172.21.15.98:0/4004299837 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bba0 con 0x7fdfec06f310 2023-10-06T01:41:40.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 -- 172.21.15.98:0/4004299837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec06f310 msgr2=0x7fdfec15adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/4004299837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec06f310 0x7fdfec15adf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 -- 172.21.15.98:0/4004299837 shutdown_connections 2023-10-06T01:41:40.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/4004299837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec06f310 0x7fdfec15adf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.073 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 -- 172.21.15.98:0/4004299837 >> 172.21.15.98:0/4004299837 conn(0x7fdfec0680c0 msgr2=0x7fdfec0cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 -- 172.21.15.98:0/4004299837 shutdown_connections 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.069+0000 7fdff0cb4700 1 -- 172.21.15.98:0/4004299837 wait complete. 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 Processor -- start 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 -- start start 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec06f310 0x7fdfec156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1407b0 0x7fdfec0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0c7fc0 con 0x7fdfec0c75e0 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0c5190 con 0x7fdfec1407b0 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdff0cb4700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0c4580 con 0x7fdfec06f310 2023-10-06T01:41:40.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdfebfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdfebfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41378/0 (socket says 172.21.15.98:41378) 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.071+0000 7fdfebfff700 1 -- 172.21.15.98:0/2652331430 learned_addr learned my addr 172.21.15.98:0/2652331430 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfeaffd700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1407b0 0x7fdfec0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfeb7fe700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec06f310 0x7fdfec156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 -- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec06f310 msgr2=0x7fdfec156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec06f310 0x7fdfec156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 -- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1407b0 msgr2=0x7fdfec0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1407b0 0x7fdfec0c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 -- 172.21.15.98:0/2652331430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009420 con 0x7fdfec0c75e0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfebfff700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40093f0 tx=0x7fdfe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bb60 con 0x7fdfec0c75e0 2023-10-06T01:41:40.075 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4020070 con 0x7fdfec0c75e0 2023-10-06T01:41:40.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec0c4860 con 0x7fdfec0c75e0 2023-10-06T01:41:40.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401b6a0 con 0x7fdfec0c75e0 2023-10-06T01:41:40.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.072+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec0c4e20 con 0x7fdfec0c75e0 2023-10-06T01:41:40.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.073+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec05f270 con 0x7fdfec0c75e0 2023-10-06T01:41:40.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.073+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfe4022030 con 0x7fdfec0c75e0 2023-10-06T01:41:40.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.074+0000 7fdfd37fe700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 0x7fdfd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.074+0000 7fdfeb7fe700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 0x7fdfd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.074+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fdfe4086680 con 0x7fdfec0c75e0 2023-10-06T01:41:40.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.074+0000 7fdfeb7fe700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 0x7fdfd80693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ae630 tx=0x7fdfdc009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.078+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfe4053850 con 0x7fdfec0c75e0 2023-10-06T01:41:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.266+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfec05f270 con 0x7fdfec0c75e0 2023-10-06T01:41:40.269 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.267+0000 7fdfd37fe700 1 -- 172.21.15.98:0/2652331430 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14212 (secure 0 0 0) 0x7fdfe4056e70 con 0x7fdfec0c75e0 2023-10-06T01:41:40.269 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:40.270 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":50,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:37.153897+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/1433484145":"2023-10-07T01:41:34.146456+0000","172.21.15.98:6836/1433484145":"2023-10-07T01:41:34.146456+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:40.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 msgr2=0x7fdfd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 0x7fdfd80693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ae630 tx=0x7fdfdc009780 comp rx=0 tx=0).stop 2023-10-06T01:41:40.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 msgr2=0x7fdfec0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.272 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40093f0 tx=0x7fdfe4016000 comp rx=0 tx=0).stop 2023-10-06T01:41:40.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 shutdown_connections 2023-10-06T01:41:40.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fdfd8066f30 0x7fdfd80693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfec0c75e0 0x7fdfec0c7a80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.273 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfec06f310 0x7fdfec156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 --2- 172.21.15.98:0/2652331430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec1407b0 0x7fdfec0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 >> 172.21.15.98:0/2652331430 conn(0x7fdfec0680c0 msgr2=0x7fdfec0c0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 shutdown_connections 2023-10-06T01:41:40.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.269+0000 7fdff0cb4700 1 -- 172.21.15.98:0/2652331430 wait complete. 2023-10-06T01:41:40.282 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T01:41:40.282 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:40.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.396+0000 7f8a0b017700 1 Processor -- start 2023-10-06T01:41:40.397 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.396+0000 7f8a0b017700 1 -- start start 2023-10-06T01:41:40.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a0b017700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f8a040b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a0b017700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040cb7e0 con 0x7f8a040b3610 2023-10-06T01:41:40.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a0b017700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040b08a0 con 0x7f8a040b2140 2023-10-06T01:41:40.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a0b017700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040cb190 con 0x7f8a04159480 2023-10-06T01:41:40.403 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a09814700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8a040b3610 0x7f8a040cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47940/0 (socket says 172.21.15.98:47940) 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a09814700 1 -- 172.21.15.98:0/2281425365 learned_addr learned my addr 172.21.15.98:0/2281425365 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938788665 0 0) 0x7f8a040cb7e0 con 0x7f8a040b3610 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a0a015700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f8a040b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.397+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f0003570 con 0x7f8a040b3610 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065297120 0 0) 0x7f89f0003570 con 0x7f8a040b3610 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 msgr2=0x7f8a040b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f8a040b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.404 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 >> v1:172.21.15.184:6789/0 conn(0x7f8a040b2140 legacy=0x7f8a040b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040cb470 con 0x7f8a040b3610 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f89f4001470 con 0x7f8a040b3610 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 >> v1:172.21.15.98:6789/0 conn(0x7f8a040b3610 legacy=0x7f8a040cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f89f0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f0004010 0x7f89f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89f0004d50 0x7f89f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040cb190 con 0x7f89f0004d50 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000bc70 con 0x7f89f0004010 2023-10-06T01:41:40.405 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000be30 con 0x7f8a04159480 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89f0004d50 0x7f89f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a015700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f89f0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09814700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f0004010 0x7f89f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 -- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 msgr2=0x7f89f0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a04159480 0x7f89f0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 -- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f0004010 msgr2=0x7f89f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f0004010 0x7f89f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 -- 172.21.15.98:0/2281425365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0003570 con 0x7f89f0004d50 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a0a816700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89f0004d50 0x7f89f00051c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89fc001f90 tx=0x7f89fc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed98bbf7d693b67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.406 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc001950 con 0x7f89f0004d50 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc00c8b0 con 0x7f89f0004d50 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.398+0000 7f8a09013700 1 -- 172.21.15.98:0/2281425365 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00cc70 con 0x7f89f0004d50 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 -- 172.21.15.98:0/2281425365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89f0004d50 msgr2=0x7f89f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 --2- 172.21.15.98:0/2281425365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89f0004d50 0x7f89f00051c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89fc001f90 tx=0x7f89fc001640 comp rx=0 tx=0).stop 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0a816700 1 -- 172.21.15.98:0/2281425365 reap_dead start 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 -- 172.21.15.98:0/2281425365 shutdown_connections 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 -- 172.21.15.98:0/2281425365 >> 172.21.15.98:0/2281425365 conn(0x7f8a0406ed20 msgr2=0x7f8a0406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 -- 172.21.15.98:0/2281425365 shutdown_connections 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.399+0000 7f8a0b017700 1 -- 172.21.15.98:0/2281425365 wait complete. 2023-10-06T01:41:40.407 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.400+0000 7f8a0b017700 1 Processor -- start 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.400+0000 7f8a0b017700 1 -- start start 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f0004d50 0x7f8a040c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0415ad70 0x7f8a0415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0415bb10 con 0x7f8a040b0460 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0415bdf0 con 0x7f8a0415ad70 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0415c0d0 con 0x7f89f0004d50 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41386/0 (socket says 172.21.15.98:41386) 2023-10-06T01:41:40.408 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 -- 172.21.15.98:0/4029745324 learned_addr learned my addr 172.21.15.98:0/4029745324 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0a816700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0415ad70 0x7f8a0415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0a015700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f0004d50 0x7f8a040c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 -- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f0004d50 msgr2=0x7f8a040c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f0004d50 0x7f8a040c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 -- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0415ad70 msgr2=0x7f8a0415b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0415ad70 0x7f8a0415b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 -- 172.21.15.98:0/4029745324 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0415c3b0 con 0x7f8a040b0460 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a09814700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89f40010e0 tx=0x7f89f4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400f6e0 con 0x7f8a040b0460 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f400f860 con 0x7f8a040b0460 2023-10-06T01:41:40.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400fc20 con 0x7f8a040b0460 2023-10-06T01:41:40.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0415b720 con 0x7f8a040b0460 2023-10-06T01:41:40.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.401+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a040c9810 con 0x7f8a040b0460 2023-10-06T01:41:40.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.402+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0415c540 con 0x7f8a040b0460 2023-10-06T01:41:40.410 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.403+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89f401d460 con 0x7f8a040b0460 2023-10-06T01:41:40.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.403+0000 7f89fa7fc700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.403+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f89f4084660 con 0x7f8a040b0460 2023-10-06T01:41:40.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.403+0000 7f8a0a015700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.403+0000 7f8a0a015700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a0416a300 tx=0x7f8a000091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.411 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.407+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f40518d0 con 0x7f8a040b0460 2023-10-06T01:41:40.585 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.584+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a0405f270 con 0x7f89e8066f30 2023-10-06T01:41:40.587 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.587+0000 7f89fa7fc700 1 -- 172.21.15.98:0/4029745324 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73322 (secure 0 0 0) 0x7f8a0405f270 con 0x7f89e8066f30 2023-10-06T01:41:40.588 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 msgr2=0x7f89e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a0416a300 tx=0x7f8a000091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 msgr2=0x7f8a040c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.590 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89f40010e0 tx=0x7f89f4001e50 comp rx=0 tx=0).stop 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 shutdown_connections 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0415ad70 0x7f8a0415b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f0004d50 0x7f8a040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 --2- 172.21.15.98:0/4029745324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a040b0460 0x7f8a040c7850 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 >> 172.21.15.98:0/4029745324 conn(0x7f8a0406ed20 msgr2=0x7f8a040b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 shutdown_connections 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.589+0000 7f8a0b017700 1 -- 172.21.15.98:0/4029745324 wait complete. 2023-10-06T01:41:40.591 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:40.602 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-06T01:41:39.809578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2392204,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752582516,"statfs":{"total":773094113280,"available":770644496384,"internally_reserved":0,"allocated":14233600,"data_stored":11755935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321510,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718933"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":41,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.151513+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:34.151513+0000","last_peered":"2023-10-06T01:41:34.151513+0000","last_clean":"2023-10-06T01:41:34.151513+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:34.151513+0000","last_undegraded":"2023-10-06T01:41:34.151513+0000","last_fullsized":"2023-10-06T01:41:34.151513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.968736+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:34.968736+0000","last_peered":"2023-10-06T01:41:34.968736+0000","last_clean":"2023-10-06T01:41:34.968736+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:34.968736+0000","last_undegraded":"2023-10-06T01:41:34.968736+0000","last_fullsized":"2023-10-06T01:41:34.968736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.150500+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:34.150500+0000","last_peered":"2023-10-06T01:41:34.150500+0000","last_clean":"2023-10-06T01:41:34.150500+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:34.150500+0000","last_undegraded":"2023-10-06T01:41:34.150500+0000","last_fullsized":"2023-10-06T01:41:34.150500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":187,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275257+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:36.275257+0000","last_peered":"2023-10-06T01:41:36.275257+0000","last_clean":"2023-10-06T01:41:36.275257+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:36.275257+0000","last_undegraded":"2023-10-06T01:41:36.275257+0000","last_fullsized":"2023-10-06T01:41:36.275257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.276533+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:36.276533+0000","last_peered":"2023-10-06T01:41:36.276533+0000","last_clean":"2023-10-06T01:41:36.276533+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:36.276533+0000","last_undegraded":"2023-10-06T01:41:36.276533+0000","last_fullsized":"2023-10-06T01:41:36.276533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275156+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:36.275156+0000","last_peered":"2023-10-06T01:41:36.275156+0000","last_clean":"2023-10-06T01:41:36.275156+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:36.275156+0000","last_undegraded":"2023-10-06T01:41:36.275156+0000","last_fullsized":"2023-10-06T01:41:36.275156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:35.280982+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:35.280982+0000","last_peered":"2023-10-06T01:41:35.280982+0000","last_clean":"2023-10-06T01:41:35.280982+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:35.280982+0000","last_undegraded":"2023-10-06T01:41:35.280982+0000","last_fullsized":"2023-10-06T01:41:35.280982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.158377+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:36.158377+0000","last_peered":"2023-10-06T01:41:36.158377+0000","last_clean":"2023-10-06T01:41:36.158377+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:36.158377+0000","last_undegraded":"2023-10-06T01:41:36.158377+0000","last_fullsized":"2023-10-06T01:41:36.158377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.159724+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:36.159724+0000","last_peered":"2023-10-06T01:41:36.159724+0000","last_clean":"2023-10-06T01:41:36.159724+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:36.159724+0000","last_undegraded":"2023-10-06T01:41:36.159724+0000","last_fullsized":"2023-10-06T01:41:36.159724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.314},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":4664,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":4775936,"data_stored":4467778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.25},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.626}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":272121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.218},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.317}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.48899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":273474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299904,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071936,"statfs":{"total":96636764160,"available":96329662464,"internally_reserved":0,"allocated":2678784,"data_stored":2370626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.628}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.499},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071944,"statfs":{"total":96636764160,"available":96329670656,"internally_reserved":0,"allocated":2670592,"data_stored":2364427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.374},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.627},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:40.604 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.717+0000 7f9395f50700 1 Processor -- start 2023-10-06T01:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.717+0000 7f9395f50700 1 -- start start 2023-10-06T01:41:40.721 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f9395f50700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93900cd480 0x7f93901645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f9395f50700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900b0de0 con 0x7f93900b29f0 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f9395f50700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900cbfe0 con 0x7f93900b2620 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f9395f50700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900cb3d0 con 0x7f93900cd480 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938effd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f93900b29f0 0x7f9390159160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47950/0 (socket says 172.21.15.98:47950) 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938effd700 1 -- 172.21.15.98:0/1074897999 learned_addr learned my addr 172.21.15.98:0/1074897999 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894102069 0 0) 0x7f9384001110 con 0x7f93900b29f0 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9378003570 con 0x7f93900b29f0 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938ffff700 1 --2- 172.21.15.98:0/1074897999 >> v2:172.21.15.184:3300/0 conn(0x7f93900cd480 0x7f93901645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.718+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499497223 0 0) 0x7f9378003570 con 0x7f93900b29f0 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 >> v2:172.21.15.184:3300/0 conn(0x7f93900cd480 msgr2=0x7f93901645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 --2- 172.21.15.98:0/1074897999 >> v2:172.21.15.184:3300/0 conn(0x7f93900cd480 0x7f93901645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 >> v1:172.21.15.184:6789/0 conn(0x7f93900b2620 legacy=0x7f93900b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900cb6b0 con 0x7f93900b29f0 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f9384001600 con 0x7f93900b29f0 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 >> v1:172.21.15.98:6789/0 conn(0x7f93900b29f0 legacy=0x7f9390159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378003a60 0x7f9378003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 0x7f9378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93780051c0 0x7f937800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900cb3d0 con 0x7f9378004810 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937800c160 con 0x7f93780051c0 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937800c320 con 0x7f9378003a60 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 0x7f9378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.723 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938f7fe700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378003a60 0x7f9378003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938ffff700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93780051c0 0x7f937800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 -- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378003a60 msgr2=0x7f9378003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378003a60 0x7f9378003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 -- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93780051c0 msgr2=0x7f937800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93780051c0 0x7f937800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938ffff700 1 -- 172.21.15.98:0/1074897999 reap_dead start 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 -- 172.21.15.98:0/1074897999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378003570 con 0x7f9378004810 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938effd700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 0x7f9378004c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f938400c5c0 tx=0x7f9384007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7e4723c4e97351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938400cd40 con 0x7f9378004810 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384005920 con 0x7f9378004810 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.719+0000 7f938e7fc700 1 -- 172.21.15.98:0/1074897999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384005ce0 con 0x7f9378004810 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 -- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 msgr2=0x7f9378004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 0x7f9378004c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f938400c5c0 tx=0x7f9384007a50 comp rx=0 tx=0).stop 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 -- 172.21.15.98:0/1074897999 shutdown_connections 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 --2- 172.21.15.98:0/1074897999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9378004810 0x7f9378004c80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 -- 172.21.15.98:0/1074897999 >> 172.21.15.98:0/1074897999 conn(0x7f939006ed20 msgr2=0x7f939006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 -- 172.21.15.98:0/1074897999 shutdown_connections 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.720+0000 7f9395f50700 1 -- 172.21.15.98:0/1074897999 wait complete. 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.721+0000 7f9395f50700 1 Processor -- start 2023-10-06T01:41:40.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.721+0000 7f9395f50700 1 -- start start 2023-10-06T01:41:40.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378004810 0x7f939014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.727 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9390152180 0x7f93901525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939014e790 con 0x7f939014b510 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9390153ac0 con 0x7f9390152180 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f9395f50700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9390152eb0 con 0x7f9378004810 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41406/0 (socket says 172.21.15.98:41406) 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 -- 172.21.15.98:0/1228262397 learned_addr learned my addr 172.21.15.98:0/1228262397 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938f7fe700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378004810 0x7f939014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938ffff700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9390152180 0x7f93901525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 -- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378004810 msgr2=0x7f939014e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378004810 0x7f939014e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 -- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9390152180 msgr2=0x7f93901525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.728 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9390152180 0x7f93901525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 -- 172.21.15.98:0/1228262397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390153190 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.722+0000 7f938effd700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f937800c130 tx=0x7f9384015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.723+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384001d20 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.723+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384005ca0 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.723+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9390153470 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.723+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384007530 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.723+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939014bec0 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.724+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9390153c30 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.724+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f938401e430 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.724+0000 7f93777fe700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 0x7f937c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.725+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f9384084ca0 con 0x7f939014b510 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.725+0000 7f938f7fe700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 0x7f937c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:40.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.725+0000 7f938f7fe700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 0x7f937c0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9380000f80 tx=0x7f9380001a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:40.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.729+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9384051f10 con 0x7f939014b510 2023-10-06T01:41:40.906 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.906+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93900b0de0 con 0x7f937c066f30 2023-10-06T01:41:40.909 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.909+0000 7f93777fe700 1 -- 172.21.15.98:0/1228262397 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73322 (secure 0 0 0) 0x7f93900b0de0 con 0x7f937c066f30 2023-10-06T01:41:40.910 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:40.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.910+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 msgr2=0x7f937c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.910+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 0x7f937c0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9380000f80 tx=0x7f9380001a70 comp rx=0 tx=0).stop 2023-10-06T01:41:40.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.910+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 msgr2=0x7f939014b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:40.911 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.910+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f937800c130 tx=0x7f9384015000 comp rx=0 tx=0).stop 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.910+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 shutdown_connections 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f937c066f30 0x7f937c0693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9390152180 0x7f93901525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9378004810 0x7f939014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 --2- 172.21.15.98:0/1228262397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f939014b510 0x7f939014b980 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 >> 172.21.15.98:0/1228262397 conn(0x7f939006ed20 msgr2=0x7f939015fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 shutdown_connections 2023-10-06T01:41:40.912 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:40.911+0000 7f9395f50700 1 -- 172.21.15.98:0/1228262397 wait complete. 2023-10-06T01:41:40.913 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:40.923 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-06T01:41:39.809578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2392204,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752582516,"statfs":{"total":773094113280,"available":770644496384,"internally_reserved":0,"allocated":14233600,"data_stored":11755935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321510,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718933"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":41,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.151513+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:34.151513+0000","last_peered":"2023-10-06T01:41:34.151513+0000","last_clean":"2023-10-06T01:41:34.151513+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:34.151513+0000","last_undegraded":"2023-10-06T01:41:34.151513+0000","last_fullsized":"2023-10-06T01:41:34.151513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.968736+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:34.968736+0000","last_peered":"2023-10-06T01:41:34.968736+0000","last_clean":"2023-10-06T01:41:34.968736+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:34.968736+0000","last_undegraded":"2023-10-06T01:41:34.968736+0000","last_fullsized":"2023-10-06T01:41:34.968736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.150500+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:34.150500+0000","last_peered":"2023-10-06T01:41:34.150500+0000","last_clean":"2023-10-06T01:41:34.150500+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:34.150500+0000","last_undegraded":"2023-10-06T01:41:34.150500+0000","last_fullsized":"2023-10-06T01:41:34.150500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":187,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275257+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:36.275257+0000","last_peered":"2023-10-06T01:41:36.275257+0000","last_clean":"2023-10-06T01:41:36.275257+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:36.275257+0000","last_undegraded":"2023-10-06T01:41:36.275257+0000","last_fullsized":"2023-10-06T01:41:36.275257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.276533+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:36.276533+0000","last_peered":"2023-10-06T01:41:36.276533+0000","last_clean":"2023-10-06T01:41:36.276533+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:36.276533+0000","last_undegraded":"2023-10-06T01:41:36.276533+0000","last_fullsized":"2023-10-06T01:41:36.276533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275156+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:36.275156+0000","last_peered":"2023-10-06T01:41:36.275156+0000","last_clean":"2023-10-06T01:41:36.275156+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:36.275156+0000","last_undegraded":"2023-10-06T01:41:36.275156+0000","last_fullsized":"2023-10-06T01:41:36.275156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:35.280982+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:35.280982+0000","last_peered":"2023-10-06T01:41:35.280982+0000","last_clean":"2023-10-06T01:41:35.280982+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:35.280982+0000","last_undegraded":"2023-10-06T01:41:35.280982+0000","last_fullsized":"2023-10-06T01:41:35.280982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.158377+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:36.158377+0000","last_peered":"2023-10-06T01:41:36.158377+0000","last_clean":"2023-10-06T01:41:36.158377+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:36.158377+0000","last_undegraded":"2023-10-06T01:41:36.158377+0000","last_fullsized":"2023-10-06T01:41:36.158377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.159724+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:36.159724+0000","last_peered":"2023-10-06T01:41:36.159724+0000","last_clean":"2023-10-06T01:41:36.159724+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:36.159724+0000","last_undegraded":"2023-10-06T01:41:36.159724+0000","last_fullsized":"2023-10-06T01:41:36.159724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.314},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":4664,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":4775936,"data_stored":4467778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.25},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.626}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":272121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.218},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.317}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.48899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":273474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299904,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071936,"statfs":{"total":96636764160,"available":96329662464,"internally_reserved":0,"allocated":2678784,"data_stored":2370626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.628}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.499},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071944,"statfs":{"total":96636764160,"available":96329670656,"internally_reserved":0,"allocated":2670592,"data_stored":2364427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.374},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.627},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:40.925 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T01:41:40.926 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.040+0000 7f38dca54700 1 Processor -- start 2023-10-06T01:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.040+0000 7f38dca54700 1 -- start start 2023-10-06T01:41:41.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38dca54700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 0x7f38d80aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.043 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38dca54700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f38d80b07a0 0x7f38d80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38dca54700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d80b0ca0 con 0x7f38d80b49e0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38dca54700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80b0f80 con 0x7f38d80b07a0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38dca54700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b1260 con 0x7f38d81595a0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d5d9b700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f38d80b49e0 0x7f38d80aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47962/0 (socket says 172.21.15.98:47962) 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d5d9b700 1 -- 172.21.15.98:0/137923664 learned_addr learned my addr 172.21.15.98:0/137923664 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/137923664 >> v2:172.21.15.184:3300/0 conn(0x7f38d80b07a0 0x7f38d80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d659c700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 0x7f38d80aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d559a700 1 -- 172.21.15.98:0/137923664 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935633527 0 0) 0x7f38d80b0ca0 con 0x7f38d80b49e0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d559a700 1 -- 172.21.15.98:0/137923664 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c0003570 con 0x7f38d80b49e0 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d6d9d700 1 -- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 msgr2=0x7f38d80aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 0x7f38d80aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d6d9d700 1 -- 172.21.15.98:0/137923664 >> v1:172.21.15.98:6789/0 conn(0x7f38d80b49e0 legacy=0x7f38d80aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d6d9d700 1 -- 172.21.15.98:0/137923664 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80b1540 con 0x7f38d80b07a0 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.041+0000 7f38d659c700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 0x7f38d80aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b07a0 0x7f38d80b1c00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009e60 tx=0x7f38d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeda22922ac174bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38d559a700 1 -- 172.21.15.98:0/137923664 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000e070 con 0x7f38d80b07a0 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38d559a700 1 -- 172.21.15.98:0/137923664 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000bab0 con 0x7f38d80b07a0 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38d559a700 1 -- 172.21.15.98:0/137923664 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000be70 con 0x7f38d80b07a0 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 -- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b07a0 msgr2=0x7f38d80b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b07a0 0x7f38d80b1c00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009e60 tx=0x7f38d0009780 comp rx=0 tx=0).stop 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 -- 172.21.15.98:0/137923664 shutdown_connections 2023-10-06T01:41:41.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d81595a0 0x7f38d80aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 --2- 172.21.15.98:0/137923664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b07a0 0x7f38d80b1c00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 -- 172.21.15.98:0/137923664 >> 172.21.15.98:0/137923664 conn(0x7f38d8099830 msgr2=0x7f38d806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:41.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 -- 172.21.15.98:0/137923664 shutdown_connections 2023-10-06T01:41:41.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.042+0000 7f38dca54700 1 -- 172.21.15.98:0/137923664 wait complete. 2023-10-06T01:41:41.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.043+0000 7f38dca54700 1 Processor -- start 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.043+0000 7f38dca54700 1 -- start start 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d80b07a0 0x7f38d81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b49e0 0x7f38d80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ab3e0 con 0x7f38d81595a0 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ab680 con 0x7f38d80b49e0 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38dca54700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ab960 con 0x7f38d80b07a0 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38d6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38d6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41408/0 (socket says 172.21.15.98:41408) 2023-10-06T01:41:41.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38d6d9d700 1 -- 172.21.15.98:0/2236405742 learned_addr learned my addr 172.21.15.98:0/2236405742 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38d5d9b700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b49e0 0x7f38d80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.044+0000 7f38d659c700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d80b07a0 0x7f38d81582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 -- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d80b07a0 msgr2=0x7f38d81582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d80b07a0 0x7f38d81582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 -- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b49e0 msgr2=0x7f38d80aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b49e0 0x7f38d80aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 -- 172.21.15.98:0/2236405742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80093c0 con 0x7f38d81595a0 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38d6d9d700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f38d8067c60 tx=0x7f38d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0012e10 con 0x7f38d81595a0 2023-10-06T01:41:41.050 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0027470 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d0009a20 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0027830 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.045+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80abf20 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.046+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d8064ad0 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.046+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38d0024070 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.046+0000 7f38c67fc700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 0x7f38b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.047+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f38d0086bb0 con 0x7f38d81595a0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.047+0000 7f38d659c700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 0x7f38b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.051 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.047+0000 7f38d659c700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 0x7f38b80693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009e90 tx=0x7f38c8009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.052 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.051+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38d0053e20 con 0x7f38d81595a0 2023-10-06T01:41:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.233+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38d80b1820 con 0x7f38b8066f30 2023-10-06T01:41:41.236 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.236+0000 7f38c67fc700 1 -- 172.21.15.98:0/2236405742 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73322 (secure 0 0 0) 0x7f38d80b1820 con 0x7f38b8066f30 2023-10-06T01:41:41.237 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:41.238 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 msgr2=0x7f38b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 0x7f38b80693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009e90 tx=0x7f38c8009120 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 msgr2=0x7f38d809d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f38d8067c60 tx=0x7f38d0019000 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 shutdown_connections 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f38b8066f30 0x7f38b80693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38d81595a0 0x7f38d809d5c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38d80b07a0 0x7f38d81582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 --2- 172.21.15.98:0/2236405742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38d80b49e0 0x7f38d80aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 >> 172.21.15.98:0/2236405742 conn(0x7f38d8099830 msgr2=0x7f38d8068400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:41.239 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 shutdown_connections 2023-10-06T01:41:41.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.238+0000 7f38dca54700 1 -- 172.21.15.98:0/2236405742 wait complete. 2023-10-06T01:41:41.240 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:41.250 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-06T01:41:39.809578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2392204,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752582516,"statfs":{"total":773094113280,"available":770644496384,"internally_reserved":0,"allocated":14233600,"data_stored":11755935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321510,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718933"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":41,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.151513+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:34.151513+0000","last_peered":"2023-10-06T01:41:34.151513+0000","last_clean":"2023-10-06T01:41:34.151513+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:34.151513+0000","last_undegraded":"2023-10-06T01:41:34.151513+0000","last_fullsized":"2023-10-06T01:41:34.151513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.968736+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:34.968736+0000","last_peered":"2023-10-06T01:41:34.968736+0000","last_clean":"2023-10-06T01:41:34.968736+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:34.968736+0000","last_undegraded":"2023-10-06T01:41:34.968736+0000","last_fullsized":"2023-10-06T01:41:34.968736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:34.150500+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:34.150500+0000","last_peered":"2023-10-06T01:41:34.150500+0000","last_clean":"2023-10-06T01:41:34.150500+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:34.150500+0000","last_undegraded":"2023-10-06T01:41:34.150500+0000","last_fullsized":"2023-10-06T01:41:34.150500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":187,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275257+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:36.275257+0000","last_peered":"2023-10-06T01:41:36.275257+0000","last_clean":"2023-10-06T01:41:36.275257+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:36.275257+0000","last_undegraded":"2023-10-06T01:41:36.275257+0000","last_fullsized":"2023-10-06T01:41:36.275257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.276533+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:36.276533+0000","last_peered":"2023-10-06T01:41:36.276533+0000","last_clean":"2023-10-06T01:41:36.276533+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:36.276533+0000","last_undegraded":"2023-10-06T01:41:36.276533+0000","last_fullsized":"2023-10-06T01:41:36.276533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275156+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:36.275156+0000","last_peered":"2023-10-06T01:41:36.275156+0000","last_clean":"2023-10-06T01:41:36.275156+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:36.275156+0000","last_undegraded":"2023-10-06T01:41:36.275156+0000","last_fullsized":"2023-10-06T01:41:36.275156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T01:41:35.280982+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:35.280982+0000","last_peered":"2023-10-06T01:41:35.280982+0000","last_clean":"2023-10-06T01:41:35.280982+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:35.280982+0000","last_undegraded":"2023-10-06T01:41:35.280982+0000","last_fullsized":"2023-10-06T01:41:35.280982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.158377+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:36.158377+0000","last_peered":"2023-10-06T01:41:36.158377+0000","last_clean":"2023-10-06T01:41:36.158377+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:36.158377+0000","last_undegraded":"2023-10-06T01:41:36.158377+0000","last_fullsized":"2023-10-06T01:41:36.158377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.159724+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:36.159724+0000","last_peered":"2023-10-06T01:41:36.159724+0000","last_clean":"2023-10-06T01:41:36.159724+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:36.159724+0000","last_undegraded":"2023-10-06T01:41:36.159724+0000","last_fullsized":"2023-10-06T01:41:36.159724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9820000000000002,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.6619999999999999,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9059999999999997,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":6.8019999999999996,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":7.6500000000000004,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":863842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.314},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":4664,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":4775936,"data_stored":4467778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.25},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.626}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":272121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":7.3449999999999998,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.218},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.317}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4130000000000003,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.4710000000000001,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.48899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":273474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299904,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071936,"statfs":{"total":96636764160,"available":96329662464,"internally_reserved":0,"allocated":2678784,"data_stored":2370626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.3460000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.628}]},{"osd":3,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.452,"5min":7.452,"15min":7.452},"last":0.499},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.476,"5min":7.476,"15min":7.476},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071944,"statfs":{"total":96636764160,"available":96329670656,"internally_reserved":0,"allocated":2670592,"data_stored":2364427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.374},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.627},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:41.253 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T01:41:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.367+0000 7fa809313700 1 Processor -- start 2023-10-06T01:41:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.367+0000 7fa809313700 1 -- start start 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa809313700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa804067840 0x7fa80406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa809313700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa809313700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040cc880 con 0x7fa804067c10 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa809313700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8040c4060 con 0x7fa804067840 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa809313700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040ccb00 con 0x7fa8040cad90 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa8037fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa804067c10 0x7fa8040c1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47970/0 (socket says 172.21.15.98:47970) 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa8037fe700 1 -- 172.21.15.98:0/2599099098 learned_addr learned my addr 172.21.15.98:0/2599099098 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa802ffd700 1 --2- 172.21.15.98:0/2599099098 >> v2:172.21.15.184:3300/0 conn(0x7fa804067840 0x7fa80406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.368+0000 7fa8027fc700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.371 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa801ffb700 1 -- 172.21.15.98:0/2599099098 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443850274 0 0) 0x7fa8040cc880 con 0x7fa804067c10 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa801ffb700 1 -- 172.21.15.98:0/2599099098 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec003570 con 0x7fa804067c10 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa802ffd700 1 -- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 msgr2=0x7fa8040c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa802ffd700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa802ffd700 1 -- 172.21.15.98:0/2599099098 >> v1:172.21.15.98:6789/0 conn(0x7fa804067c10 legacy=0x7fa8040c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa802ffd700 1 -- 172.21.15.98:0/2599099098 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040ccde0 con 0x7fa804067840 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa8027fc700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa802ffd700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067840 0x7fa80406eed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40098e0 tx=0x7fa7f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4566e0b134b8d3f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa801ffb700 1 -- 172.21.15.98:0/2599099098 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4011630 con 0x7fa804067840 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa801ffb700 1 -- 172.21.15.98:0/2599099098 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f4011790 con 0x7fa804067840 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.369+0000 7fa801ffb700 1 -- 172.21.15.98:0/2599099098 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4011b30 con 0x7fa804067840 2023-10-06T01:41:41.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 -- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067840 msgr2=0x7fa80406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067840 0x7fa80406eed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40098e0 tx=0x7fa7f40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 -- 172.21.15.98:0/2599099098 shutdown_connections 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067840 0x7fa80406eed0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 --2- 172.21.15.98:0/2599099098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 -- 172.21.15.98:0/2599099098 >> 172.21.15.98:0/2599099098 conn(0x7fa8040691c0 msgr2=0x7fa8040cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 -- 172.21.15.98:0/2599099098 shutdown_connections 2023-10-06T01:41:41.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.370+0000 7fa809313700 1 -- 172.21.15.98:0/2599099098 wait complete. 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.371+0000 7fa809313700 1 Processor -- start 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.371+0000 7fa809313700 1 -- start start 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 0x7fa80414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804154190 con 0x7fa804067840 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804153580 con 0x7fa804067c10 2023-10-06T01:41:41.376 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa804153860 con 0x7fa8040cad90 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41414/0 (socket says 172.21.15.98:41414) 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 -- 172.21.15.98:0/1825145500 learned_addr learned my addr 172.21.15.98:0/1825145500 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa8037fe700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa8027fc700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 0x7fa80414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 -- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 msgr2=0x7fa8040b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 -- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 msgr2=0x7fa80414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 0x7fa80414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 -- 172.21.15.98:0/1825145500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009420 con 0x7fa804067840 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa8027fc700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 0x7fa80414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:41.377 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa802ffd700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8040713d0 tx=0x7fa7f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4011b10 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40253f0 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f4009420 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.372+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40257b0 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.373+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804153e20 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.373+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804154470 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.374+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7f4025ba0 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.374+0000 7fa7eaffd700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 0x7fa7f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.374+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fa7f4085cd0 con 0x7fa804067840 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.374+0000 7fa8027fc700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 0x7fa7f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:41.378 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.375+0000 7fa8027fc700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 0x7fa7f00693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80093f0 tx=0x7fa7f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:41.379 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.378+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7f4052f40 con 0x7fa804067840 2023-10-06T01:41:41.563 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.563+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa804064ad0 con 0x7fa804067840 2023-10-06T01:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.563+0000 7fa7eaffd700 1 -- 172.21.15.98:0/1825145500 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14212 (secure 0 0 0) 0x7fa7f4015090 con 0x7fa804067840 2023-10-06T01:41:41.564 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:41.564 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":50,"fsid":"c6dd341a-820f-4e37-8897-3250639f93a1","created":"2023-10-06T01:39:26.248322+0000","modified":"2023-10-06T01:41:37.153897+0000","last_up_change":"2023-10-06T01:39:37.022322+0000","last_in_change":"2023-10-06T01:39:28.625965+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T01:39:37.371756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T01:39:37.826565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4844d71-6e27-46c9-9909-2d60d00e922a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":120654},{"type":"v1","addr":"172.21.15.98:6801","nonce":120654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":120654},{"type":"v1","addr":"172.21.15.98:6803","nonce":120654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":120654},{"type":"v1","addr":"172.21.15.98:6807","nonce":120654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":120654},{"type":"v1","addr":"172.21.15.98:6805","nonce":120654}]},"public_addr":"172.21.15.98:6801/120654","cluster_addr":"172.21.15.98:6803/120654","heartbeat_back_addr":"172.21.15.98:6807/120654","heartbeat_front_addr":"172.21.15.98:6805/120654","state":["exists","up"]},{"osd":1,"uuid":"77069145-8b7e-4c1f-a831-9497591970cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":120656},{"type":"v1","addr":"172.21.15.98:6810","nonce":120656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":120656},{"type":"v1","addr":"172.21.15.98:6814","nonce":120656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":120656},{"type":"v1","addr":"172.21.15.98:6822","nonce":120656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":120656},{"type":"v1","addr":"172.21.15.98:6818","nonce":120656}]},"public_addr":"172.21.15.98:6810/120656","cluster_addr":"172.21.15.98:6814/120656","heartbeat_back_addr":"172.21.15.98:6822/120656","heartbeat_front_addr":"172.21.15.98:6818/120656","state":["exists","up"]},{"osd":2,"uuid":"e6c83413-b7df-4e9a-8c56-826e9ea15c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":120679},{"type":"v1","addr":"172.21.15.98:6825","nonce":120679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":120679},{"type":"v1","addr":"172.21.15.98:6827","nonce":120679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":120679},{"type":"v1","addr":"172.21.15.98:6831","nonce":120679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":120679},{"type":"v1","addr":"172.21.15.98:6829","nonce":120679}]},"public_addr":"172.21.15.98:6825/120679","cluster_addr":"172.21.15.98:6827/120679","heartbeat_back_addr":"172.21.15.98:6831/120679","heartbeat_front_addr":"172.21.15.98:6829/120679","state":["exists","up"]},{"osd":3,"uuid":"080f0848-851e-4596-84c3-b70d766b7b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6809","nonce":120655},{"type":"v1","addr":"172.21.15.98:6811","nonce":120655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6813","nonce":120655},{"type":"v1","addr":"172.21.15.98:6815","nonce":120655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6821","nonce":120655},{"type":"v1","addr":"172.21.15.98:6823","nonce":120655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":120655},{"type":"v1","addr":"172.21.15.98:6819","nonce":120655}]},"public_addr":"172.21.15.98:6811/120655","cluster_addr":"172.21.15.98:6815/120655","heartbeat_back_addr":"172.21.15.98:6823/120655","heartbeat_front_addr":"172.21.15.98:6819/120655","state":["exists","up"]},{"osd":4,"uuid":"e213de30-3383-4cbb-9b68-1e22e089b94f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":120887},{"type":"v1","addr":"172.21.15.184:6825","nonce":120887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":120887},{"type":"v1","addr":"172.21.15.184:6827","nonce":120887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":120887},{"type":"v1","addr":"172.21.15.184:6831","nonce":120887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":120887},{"type":"v1","addr":"172.21.15.184:6829","nonce":120887}]},"public_addr":"172.21.15.184:6825/120887","cluster_addr":"172.21.15.184:6827/120887","heartbeat_back_addr":"172.21.15.184:6831/120887","heartbeat_front_addr":"172.21.15.184:6829/120887","state":["exists","up"]},{"osd":5,"uuid":"f890cdc2-7390-4a47-ac30-19ca54ed753e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":120874},{"type":"v1","addr":"172.21.15.184:6804","nonce":120874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":120874},{"type":"v1","addr":"172.21.15.184:6808","nonce":120874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":120874},{"type":"v1","addr":"172.21.15.184:6815","nonce":120874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":120874},{"type":"v1","addr":"172.21.15.184:6812","nonce":120874}]},"public_addr":"172.21.15.184:6804/120874","cluster_addr":"172.21.15.184:6808/120874","heartbeat_back_addr":"172.21.15.184:6815/120874","heartbeat_front_addr":"172.21.15.184:6812/120874","state":["exists","up"]},{"osd":6,"uuid":"7dd97ec5-9e35-4b7b-a2fe-b5996b0f2aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":120875},{"type":"v1","addr":"172.21.15.184:6801","nonce":120875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":120875},{"type":"v1","addr":"172.21.15.184:6805","nonce":120875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":120875},{"type":"v1","addr":"172.21.15.184:6813","nonce":120875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":120875},{"type":"v1","addr":"172.21.15.184:6809","nonce":120875}]},"public_addr":"172.21.15.184:6801/120875","cluster_addr":"172.21.15.184:6805/120875","heartbeat_back_addr":"172.21.15.184:6813/120875","heartbeat_front_addr":"172.21.15.184:6809/120875","state":["exists","up"]},{"osd":7,"uuid":"eda3a706-f9f2-4757-89f7-65d481de0792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":120876},{"type":"v1","addr":"172.21.15.184:6817","nonce":120876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":120876},{"type":"v1","addr":"172.21.15.184:6819","nonce":120876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":120876},{"type":"v1","addr":"172.21.15.184:6823","nonce":120876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":120876},{"type":"v1","addr":"172.21.15.184:6821","nonce":120876}]},"public_addr":"172.21.15.184:6817/120876","cluster_addr":"172.21.15.184:6819/120876","heartbeat_back_addr":"172.21.15.184:6823/120876","heartbeat_front_addr":"172.21.15.184:6821/120876","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.757788+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.791482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.820320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.744956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.873870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.773662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.715192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T01:39:33.851669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/1433484145":"2023-10-07T01:41:34.146456+0000","172.21.15.98:6836/1433484145":"2023-10-07T01:41:34.146456+0000","172.21.15.184:6833/1929325397":"2023-10-07T01:40:55.207952+0000","172.21.15.184:6835/1929325397":"2023-10-07T01:40:55.207952+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T01:41:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 msgr2=0x7fa7f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 0x7fa7f00693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80093f0 tx=0x7fa7f8008040 comp rx=0 tx=0).stop 2023-10-06T01:41:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 msgr2=0x7fa804154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8040713d0 tx=0x7fa7f4017000 comp rx=0 tx=0).stop 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 shutdown_connections 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa7f0066f30 0x7fa7f00693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8040cad90 0x7fa8040b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa804067840 0x7fa804154620 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 --2- 172.21.15.98:0/1825145500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa804067c10 0x7fa80414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 >> 172.21.15.98:0/1825145500 conn(0x7fa8040691c0 msgr2=0x7fa8040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 shutdown_connections 2023-10-06T01:41:41.567 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:41.565+0000 7fa809313700 1 -- 172.21.15.98:0/1825145500 wait complete. 2023-10-06T01:41:42.578 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T01:41:42.578 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:42.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.701+0000 7fd2dcf81700 1 Processor -- start 2023-10-06T01:41:42.702 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.701+0000 7fd2dcf81700 1 -- start start 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2dcf81700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd2d806ed20 0x7fd2d80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2dcf81700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2d806f0f0 0x7fd2d80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2dcf81700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80b2d20 con 0x7fd2d80aee20 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2dcf81700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d806fb80 con 0x7fd2d806f0f0 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2dcf81700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80b0450 con 0x7fd2d806ed20 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2d806f0f0 0x7fd2d80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d6d9d700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd2d80aee20 0x7fd2d80af280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42448/0 (socket says 172.21.15.98:42448) 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d6d9d700 1 -- 172.21.15.98:0/1082011725 learned_addr learned my addr 172.21.15.98:0/1082011725 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/1082011725 >> v2:172.21.15.184:3300/0 conn(0x7fd2d806ed20 msgr2=0x7fd2d80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:42.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 --2- 172.21.15.98:0/1082011725 >> v2:172.21.15.184:3300/0 conn(0x7fd2d806ed20 0x7fd2d80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/1082011725 >> v1:172.21.15.98:6789/0 conn(0x7fd2d80aee20 legacy=0x7fd2d80af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/1082011725 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80b2a40 con 0x7fd2d806f0f0 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d559a700 1 -- 172.21.15.98:0/1082011725 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578153385 0 0) 0x7fd2d80b2d20 con 0x7fd2d80aee20 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d5d9b700 1 --2- 172.21.15.98:0/1082011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80b08e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098e0 tx=0x7fd2cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a7e5895b129d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.702+0000 7fd2d559a700 1 -- 172.21.15.98:0/1082011725 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc009e80 con 0x7fd2d806f0f0 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2d559a700 1 -- 172.21.15.98:0/1082011725 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc00b820 con 0x7fd2d806f0f0 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2d559a700 1 -- 172.21.15.98:0/1082011725 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00bba0 con 0x7fd2d806f0f0 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 -- 172.21.15.98:0/1082011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 msgr2=0x7fd2d80b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/1082011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80b08e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098e0 tx=0x7fd2cc0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 -- 172.21.15.98:0/1082011725 shutdown_connections 2023-10-06T01:41:42.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/1082011725 >> v2:172.21.15.184:3300/0 conn(0x7fd2d806ed20 0x7fd2d80b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/1082011725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80b08e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 -- 172.21.15.98:0/1082011725 >> 172.21.15.98:0/1082011725 conn(0x7fd2d8068330 msgr2=0x7fd2d8068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:42.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 -- 172.21.15.98:0/1082011725 shutdown_connections 2023-10-06T01:41:42.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.703+0000 7fd2dcf81700 1 -- 172.21.15.98:0/1082011725 wait complete. 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.704+0000 7fd2dcf81700 1 Processor -- start 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.704+0000 7fd2dcf81700 1 -- start start 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d806ed20 0x7fd2d81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d80aee20 0x7fd2d80a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.709 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a7b30 con 0x7fd2d806f0f0 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a9700 con 0x7fd2d80aee20 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2dcf81700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a6f20 con 0x7fd2d806ed20 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51036/0 (socket says 172.21.15.98:51036) 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/532905336 learned_addr learned my addr 172.21.15.98:0/532905336 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d6d9d700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d80aee20 0x7fd2d80a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d659c700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d806ed20 0x7fd2d81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d806ed20 msgr2=0x7fd2d81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d806ed20 0x7fd2d81566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.710 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d80aee20 msgr2=0x7fd2d80a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d80aee20 0x7fd2d80a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc009420 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.705+0000 7fd2d5d9b700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80a7fc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80893c0 tx=0x7fd2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011420 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc011580 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d80a7200 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011940 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d80a77c0 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.706+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2d8064ad0 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.707+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2cc02c3b0 con 0x7fd2d806f0f0 2023-10-06T01:41:42.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.707+0000 7fd2c67fc700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd2b8066f30 0x7fd2b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd2cc085ba0 con 0x7fd2d806f0f0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2d659c700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd2b8066f30 0x7fd2b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2c67fc700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7fd2b806d8f0 0x7fd2b806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2b80703f0 con 0x7fd2b806d8f0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc08a240 con 0x7fd2d806f0f0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2d6d9d700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7fd2b806d8f0 0x7fd2b806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2d659c700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd2b8066f30 0x7fd2b80693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0001250 tx=0x7fd2d0009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.708+0000 7fd2d6d9d700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7fd2b806d8f0 0x7fd2b806fd20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.712 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.709+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== osd.0 v2:172.21.15.98:6800/120654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd2b80703f0 con 0x7fd2b806d8f0 2023-10-06T01:41:42.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 --> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd2d804bb80 con 0x7fd2b806d8f0 2023-10-06T01:41:42.722 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2c67fc700 1 -- 172.21.15.98:0/532905336 <== osd.0 v2:172.21.15.98:6800/120654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd2d804bb80 con 0x7fd2b806d8f0 2023-10-06T01:41:42.723 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:42.723 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:42.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7fd2b806d8f0 msgr2=0x7fd2b806fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6800/120654,v1:172.21.15.98:6801/120654] conn(0x7fd2b806d8f0 0x7fd2b806fd20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd2b8066f30 msgr2=0x7fd2b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd2b8066f30 0x7fd2b80693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0001250 tx=0x7fd2d0009620 comp rx=0 tx=0).stop 2023-10-06T01:41:42.724 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 msgr2=0x7fd2d80a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2dcf81700 1 --2- 172.21.15.98:0/532905336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2d806f0f0 0x7fd2d80a7fc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80893c0 tx=0x7fd2cc017000 comp rx=0 tx=0).stop 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.722+0000 7fd2d6d9d700 1 -- 172.21.15.98:0/532905336 reap_dead start 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.723+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 shutdown_connections 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.723+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 >> 172.21.15.98:0/532905336 conn(0x7fd2d8068330 msgr2=0x7fd2d80b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.723+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 shutdown_connections 2023-10-06T01:41:42.725 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.723+0000 7fd2dcf81700 1 -- 172.21.15.98:0/532905336 wait complete. 2023-10-06T01:41:42.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T01:41:42.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.849+0000 7f20704de700 1 Processor -- start 2023-10-06T01:41:42.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.849+0000 7f20704de700 1 -- start start 2023-10-06T01:41:42.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f20704de700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20680b2620 0x7f20680b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f20704de700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20680cbfe0 con 0x7f20680b29f0 2023-10-06T01:41:42.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f20704de700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20680cb3d0 con 0x7f2068159870 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f20704de700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20680cb6b0 con 0x7f20680b2620 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f206e27a700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20680b2620 0x7f20680b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f206da79700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f20680b29f0 0x7f2068159160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42454/0 (socket says 172.21.15.98:42454) 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.850+0000 7f206da79700 1 -- 172.21.15.98:0/2283394272 learned_addr learned my addr 172.21.15.98:0/2283394272 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206e27a700 1 -- 172.21.15.98:0/2283394272 >> v1:172.21.15.184:6789/0 conn(0x7f2068159870 legacy=0x7f20680cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206e27a700 1 -- 172.21.15.98:0/2283394272 >> v1:172.21.15.98:6789/0 conn(0x7f20680b29f0 legacy=0x7f2068159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206e27a700 1 -- 172.21.15.98:0/2283394272 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680cb990 con 0x7f20680b2620 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206d278700 1 -- 172.21.15.98:0/2283394272 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203670790 0 0) 0x7f20680cbfe0 con 0x7f20680b29f0 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206e27a700 1 --2- 172.21.15.98:0/2283394272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20680b2620 0x7f20680b06d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2064009eb0 tx=0x7f2064009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f55fa066df8065 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206d278700 1 -- 172.21.15.98:0/2283394272 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400e070 con 0x7f20680b2620 2023-10-06T01:41:42.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206d278700 1 -- 172.21.15.98:0/2283394272 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206400bc30 con 0x7f20680b2620 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f206d278700 1 -- 172.21.15.98:0/2283394272 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064012430 con 0x7f20680b2620 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f20704de700 1 -- 172.21.15.98:0/2283394272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20680b2620 msgr2=0x7f20680b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.851+0000 7f20704de700 1 --2- 172.21.15.98:0/2283394272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20680b2620 0x7f20680b06d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2064009eb0 tx=0x7f2064009780 comp rx=0 tx=0).stop 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.852+0000 7f20704de700 1 -- 172.21.15.98:0/2283394272 shutdown_connections 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.852+0000 7f20704de700 1 --2- 172.21.15.98:0/2283394272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20680b2620 0x7f20680b06d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.852+0000 7f20704de700 1 -- 172.21.15.98:0/2283394272 >> 172.21.15.98:0/2283394272 conn(0x7f206806ed20 msgr2=0x7f206806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.852+0000 7f20704de700 1 -- 172.21.15.98:0/2283394272 shutdown_connections 2023-10-06T01:41:42.854 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.852+0000 7f20704de700 1 -- 172.21.15.98:0/2283394272 wait complete. 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 Processor -- start 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 -- start start 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20680b2620 0x7f20680d0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20680b29f0 0x7f206816a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680c7ee0 con 0x7f2068159870 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680d0f50 con 0x7f20680b29f0 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f20704de700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20680c75b0 con 0x7f20680b2620 2023-10-06T01:41:42.857 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f206ea7b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f206ea7b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51054/0 (socket says 172.21.15.98:51054) 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.853+0000 7f206ea7b700 1 -- 172.21.15.98:0/764013900 learned_addr learned my addr 172.21.15.98:0/764013900 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206da79700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20680b29f0 0x7f206816a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206e27a700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20680b2620 0x7f20680d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 -- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20680b2620 msgr2=0x7f20680d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20680b2620 0x7f20680d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 -- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20680b29f0 msgr2=0x7f206816a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20680b29f0 0x7f206816a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:42.858 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 -- 172.21.15.98:0/764013900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009a20 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f206ea7b700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f206000c4c0 tx=0x7f2060007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011590 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060011710 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011ad0 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20680c7890 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.854+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20680d0c50 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.855+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20680c8050 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.855+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f206001b3b0 con 0x7f2068159870 2023-10-06T01:41:42.859 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.856+0000 7f205e7fc700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 0x7f204c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:42.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.856+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f20600839b0 con 0x7f2068159870 2023-10-06T01:41:42.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.856+0000 7f206e27a700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 0x7f204c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:42.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.856+0000 7f206e27a700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 0x7f204c0693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2064009eb0 tx=0x7f2064017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:42.860 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:42.860+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2060050c20 con 0x7f2068159870 2023-10-06T01:41:43.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.041+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2068064ad0 con 0x7f204c066f30 2023-10-06T01:41:43.041 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.041+0000 7f205e7fc700 1 -- 172.21.15.98:0/764013900 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2068064ad0 con 0x7f204c066f30 2023-10-06T01:41:43.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 msgr2=0x7f204c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 0x7f204c0693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2064009eb0 tx=0x7f2064017000 comp rx=0 tx=0).stop 2023-10-06T01:41:43.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 msgr2=0x7f206816ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.044 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f206000c4c0 tx=0x7f2060007f50 comp rx=0 tx=0).stop 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 shutdown_connections 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f204c066f30 0x7f204c0693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2068159870 0x7f206816ad40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20680b2620 0x7f20680d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 --2- 172.21.15.98:0/764013900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20680b29f0 0x7f206816a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 >> 172.21.15.98:0/764013900 conn(0x7f206806ed20 msgr2=0x7f20680c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 shutdown_connections 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.043+0000 7f20704de700 1 -- 172.21.15.98:0/764013900 wait complete. 2023-10-06T01:41:43.045 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T01:41:43.055 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T01:41:43.056 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:43.176 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.173+0000 7f3e0954c700 1 Processor -- start 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.173+0000 7f3e0954c700 1 -- start start 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e0954c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e0954c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e040c4b40 0x7f3e040cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e0954c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e040c4170 con 0x7f3e040b0760 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e0954c700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e040b1800 con 0x7f3e040c4b40 2023-10-06T01:41:43.177 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e0954c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c3560 con 0x7f3e040c30d0 2023-10-06T01:41:43.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e037fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e040c4b40 0x7f3e040cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e027fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e037fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e040c4b40 0x7f3e040cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.98:55192/0 (socket says 172.21.15.98:55192) 2023-10-06T01:41:43.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e027fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.98:53440/0 (socket says 172.21.15.98:53440) 2023-10-06T01:41:43.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.174+0000 7f3e037fe700 1 -- 172.21.15.98:0/1653385151 learned_addr learned my addr 172.21.15.98:0/1653385151 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e037fe700 1 -- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 msgr2=0x7f3e040c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:43.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e037fe700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e037fe700 1 -- 172.21.15.98:0/1653385151 >> v1:172.21.15.184:6789/0 conn(0x7f3e040b0760 legacy=0x7f3e040b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e037fe700 1 -- 172.21.15.98:0/1653385151 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040c3840 con 0x7f3e040c4b40 2023-10-06T01:41:43.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e027fc700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:43.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.175+0000 7f3e037fe700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040c4b40 0x7f3e040cc1b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009f20 tx=0x7f3dfc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9066c47a5712f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.176+0000 7f3e01ffb700 1 -- 172.21.15.98:0/1653385151 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00e070 con 0x7f3e040c4b40 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.176+0000 7f3e01ffb700 1 -- 172.21.15.98:0/1653385151 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00ba60 con 0x7f3e040c4b40 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.176+0000 7f3e01ffb700 1 -- 172.21.15.98:0/1653385151 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0092e0 con 0x7f3e040c4b40 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 -- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040c4b40 msgr2=0x7f3e040cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040c4b40 0x7f3e040cc1b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009f20 tx=0x7f3dfc009840 comp rx=0 tx=0).stop 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 -- 172.21.15.98:0/1653385151 shutdown_connections 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040c4b40 0x7f3e040cc1b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 --2- 172.21.15.98:0/1653385151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c30d0 0x7f3e040c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 -- 172.21.15.98:0/1653385151 >> 172.21.15.98:0/1653385151 conn(0x7f3e04088000 msgr2=0x7f3e040ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.187 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 -- 172.21.15.98:0/1653385151 shutdown_connections 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.177+0000 7f3e0954c700 1 -- 172.21.15.98:0/1653385151 wait complete. 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.178+0000 7f3e0954c700 1 Processor -- start 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.178+0000 7f3e0954c700 1 -- start start 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040b0760 0x7f3e0414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 0x7f3e040c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c4b40 0x7f3e0415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0406a7d0 con 0x7f3e040c30d0 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04069a50 con 0x7f3e040b0760 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e0954c700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c79b0 con 0x7f3e040c4b40 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 0x7f3e040c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 0x7f3e040c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51062/0 (socket says 172.21.15.98:51062) 2023-10-06T01:41:43.188 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 -- 172.21.15.98:0/381791343 learned_addr learned my addr 172.21.15.98:0/381791343 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e037fe700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c4b40 0x7f3e0415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e02ffd700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040b0760 0x7f3e0414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 -- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c4b40 msgr2=0x7f3e0415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e040c4b40 0x7f3e0415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 -- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040b0760 msgr2=0x7f3e0414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e040b0760 0x7f3e0414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df8009420 con 0x7f3e040c30d0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.179+0000 7f3e027fc700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 0x7f3e040c6ff0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3df800ec80 tx=0x7f3df80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df800c9d0 con 0x7f3e040c30d0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df800cb30 con 0x7f3e040c30d0 2023-10-06T01:41:43.189 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc009ae0 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df800ceb0 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e04069fd0 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.180+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e04064ad0 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3df8013680 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3deaffd700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3df0066f30 0x7f3df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3df8084d80 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3e02ffd700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3df0066f30 0x7f3df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3deaffd700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f3df006d870 0x7f3df006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.181+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3df0070370 con 0x7f3df006d870 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.182+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df8085150 con 0x7f3e040c30d0 2023-10-06T01:41:43.190 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.182+0000 7f3e037fe700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f3df006d870 0x7f3df006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.182+0000 7f3e02ffd700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3df0066f30 0x7f3df00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3df4000c00 tx=0x7f3df4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.182+0000 7f3e037fe700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f3df006d870 0x7f3df006fca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.191 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.182+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== osd.1 v2:172.21.15.98:6808/120656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3df0070370 con 0x7f3df006d870 2023-10-06T01:41:43.196 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 --> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3e040c3b20 con 0x7f3df006d870 2023-10-06T01:41:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3deaffd700 1 -- 172.21.15.98:0/381791343 <== osd.1 v2:172.21.15.98:6808/120656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3e040c3b20 con 0x7f3df006d870 2023-10-06T01:41:43.197 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:43.197 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:43.197 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f3df006d870 msgr2=0x7f3df006fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6808/120656,v1:172.21.15.98:6810/120656] conn(0x7f3df006d870 0x7f3df006fca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3df0066f30 msgr2=0x7f3df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3df0066f30 0x7f3df00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3df4000c00 tx=0x7f3df4009040 comp rx=0 tx=0).stop 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 msgr2=0x7f3e040c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.196+0000 7f3e0954c700 1 --2- 172.21.15.98:0/381791343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040c30d0 0x7f3e040c6ff0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3df800ec80 tx=0x7f3df80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.197+0000 7f3e037fe700 1 -- 172.21.15.98:0/381791343 reap_dead start 2023-10-06T01:41:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.197+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 shutdown_connections 2023-10-06T01:41:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.197+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 >> 172.21.15.98:0/381791343 conn(0x7f3e04088000 msgr2=0x7f3e040b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.197+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 shutdown_connections 2023-10-06T01:41:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.197+0000 7f3e0954c700 1 -- 172.21.15.98:0/381791343 wait complete. 2023-10-06T01:41:43.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T01:41:43.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.322+0000 7f3d7ea7b700 1 Processor -- start 2023-10-06T01:41:43.323 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.322+0000 7f3d7ea7b700 1 -- start start 2023-10-06T01:41:43.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7ea7b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d780b49e0 0x7f3d780aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7ea7b700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d780b0ca0 con 0x7f3d781595a0 2023-10-06T01:41:43.333 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7ea7b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d780b0f80 con 0x7f3d780b07a0 2023-10-06T01:41:43.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7ea7b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d780b1260 con 0x7f3d780b49e0 2023-10-06T01:41:43.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7c817700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3d781595a0 0x7f3d780aa570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42470/0 (socket says 172.21.15.98:42470) 2023-10-06T01:41:43.334 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d7c817700 1 -- 172.21.15.98:0/1561350742 learned_addr learned my addr 172.21.15.98:0/1561350742 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:43.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d6ffff700 1 --2- 172.21.15.98:0/1561350742 >> v2:172.21.15.184:3300/0 conn(0x7f3d780b49e0 0x7f3d780aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.335 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d6f7fe700 1 -- 172.21.15.98:0/1561350742 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858873272 0 0) 0x7f3d780b0ca0 con 0x7f3d781595a0 2023-10-06T01:41:43.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.323+0000 7f3d6f7fe700 1 -- 172.21.15.98:0/1561350742 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d60003570 con 0x7f3d781595a0 2023-10-06T01:41:43.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6ffff700 1 -- 172.21.15.98:0/1561350742 >> v1:172.21.15.184:6789/0 conn(0x7f3d780b07a0 legacy=0x7f3d780b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.336 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6ffff700 1 -- 172.21.15.98:0/1561350742 >> v1:172.21.15.98:6789/0 conn(0x7f3d781595a0 legacy=0x7f3d780aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6ffff700 1 -- 172.21.15.98:0/1561350742 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780b1540 con 0x7f3d780b49e0 2023-10-06T01:41:43.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6ffff700 1 --2- 172.21.15.98:0/1561350742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d780b49e0 0x7f3d780aac80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009910 tx=0x7f3d680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afdf64d4275456e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6f7fe700 1 -- 172.21.15.98:0/1561350742 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800b640 con 0x7f3d780b49e0 2023-10-06T01:41:43.337 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6f7fe700 1 -- 172.21.15.98:0/1561350742 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6800b7a0 con 0x7f3d780b49e0 2023-10-06T01:41:43.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d6f7fe700 1 -- 172.21.15.98:0/1561350742 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800bb60 con 0x7f3d780b49e0 2023-10-06T01:41:43.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/1561350742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d780b49e0 msgr2=0x7f3d780aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.338 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.324+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/1561350742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d780b49e0 0x7f3d780aac80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009910 tx=0x7f3d680091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.325+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/1561350742 shutdown_connections 2023-10-06T01:41:43.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.325+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/1561350742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d780b49e0 0x7f3d780aac80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.325+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/1561350742 >> 172.21.15.98:0/1561350742 conn(0x7f3d78099830 msgr2=0x7f3d7806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.325+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/1561350742 shutdown_connections 2023-10-06T01:41:43.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.325+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/1561350742 wait complete. 2023-10-06T01:41:43.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.326+0000 7f3d7ea7b700 1 Processor -- start 2023-10-06T01:41:43.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.326+0000 7f3d7ea7b700 1 -- start start 2023-10-06T01:41:43.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.341 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d780b49e0 0x7f3d780cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 0x7f3d780cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7809c490 con 0x7f3d780b07a0 2023-10-06T01:41:43.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780aea20 con 0x7f3d781595a0 2023-10-06T01:41:43.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780aed00 con 0x7f3d780b49e0 2023-10-06T01:41:43.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51074/0 (socket says 172.21.15.98:51074) 2023-10-06T01:41:43.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 -- 172.21.15.98:0/3216852110 learned_addr learned my addr 172.21.15.98:0/3216852110 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d6ffff700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d780b49e0 0x7f3d780cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7d018700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 0x7f3d780cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 -- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d780b49e0 msgr2=0x7f3d780cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d780b49e0 0x7f3d780cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 -- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 msgr2=0x7f3d780cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 0x7f3d780cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 -- 172.21.15.98:0/3216852110 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009420 con 0x7f3d780b07a0 2023-10-06T01:41:43.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7d018700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 0x7f3d780cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:43.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7c817700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c900 tx=0x7f3d74002740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400cea0 con 0x7f3d780b07a0 2023-10-06T01:41:43.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74005d80 con 0x7f3d780b07a0 2023-10-06T01:41:43.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7809be40 con 0x7f3d780b07a0 2023-10-06T01:41:43.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400a7e0 con 0x7f3d780b07a0 2023-10-06T01:41:43.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.327+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d781581c0 con 0x7f3d780b07a0 2023-10-06T01:41:43.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.328+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d78064ad0 con 0x7f3d780b07a0 2023-10-06T01:41:43.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.329+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7400ac10 con 0x7f3d780b07a0 2023-10-06T01:41:43.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.329+0000 7f3d6cff9700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.348 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.329+0000 7f3d6ffff700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.329+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3d740842c0 con 0x7f3d780b07a0 2023-10-06T01:41:43.349 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.330+0000 7f3d6ffff700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009910 tx=0x7f3d68017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.350 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.333+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d740515b0 con 0x7f3d780b07a0 2023-10-06T01:41:43.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.516+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7809c5d0 con 0x7f3d64066f30 2023-10-06T01:41:43.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.516+0000 7f3d6cff9700 1 -- 172.21.15.98:0/3216852110 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3d68002f30 con 0x7f3d64066f30 2023-10-06T01:41:43.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 msgr2=0x7f3d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.519 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009910 tx=0x7f3d68017000 comp rx=0 tx=0).stop 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 msgr2=0x7f3d78157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c900 tx=0x7f3d74002740 comp rx=0 tx=0).stop 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 shutdown_connections 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d781595a0 0x7f3d780cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d780b07a0 0x7f3d78157c80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.520 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 --2- 172.21.15.98:0/3216852110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d780b49e0 0x7f3d780cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 >> 172.21.15.98:0/3216852110 conn(0x7f3d78099830 msgr2=0x7f3d78158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 shutdown_connections 2023-10-06T01:41:43.521 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.518+0000 7f3d7ea7b700 1 -- 172.21.15.98:0/3216852110 wait complete. 2023-10-06T01:41:43.521 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T01:41:43.531 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T01:41:43.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:43.650 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.649+0000 7fb1fa922700 1 Processor -- start 2023-10-06T01:41:43.650 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.649+0000 7fb1fa922700 1 -- start start 2023-10-06T01:41:43.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1fa922700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1f406ed20 0x7fb1f40b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.660 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1fa922700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f40aee20 0x7fb1f40af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1fa922700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f40b2d20 con 0x7fb1f406f0f0 2023-10-06T01:41:43.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1fa922700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f406fb80 con 0x7fb1f406ed20 2023-10-06T01:41:43.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1fa922700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40b0450 con 0x7fb1f40aee20 2023-10-06T01:41:43.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1f406ed20 0x7fb1f40b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.661 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1f406ed20 0x7fb1f40b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51078/0 (socket says 172.21.15.98:51078) 2023-10-06T01:41:43.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 -- 172.21.15.98:0/813215035 learned_addr learned my addr 172.21.15.98:0/813215035 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f8ebf700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f40aee20 0x7fb1f40af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 -- 172.21.15.98:0/813215035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f40aee20 msgr2=0x7fb1f40af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:43.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f40aee20 0x7fb1f40af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.662 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.650+0000 7fb1f3fff700 1 -- 172.21.15.98:0/813215035 >> v1:172.21.15.184:6789/0 conn(0x7fb1f406f0f0 legacy=0x7fb1f40b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:43.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1f3fff700 1 -- 172.21.15.98:0/813215035 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40af7c0 con 0x7fb1f406ed20 2023-10-06T01:41:43.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1f3fff700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f406ed20 0x7fb1f40b23a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099b0 tx=0x7fb1e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0f783e7b031ad4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1f2ffd700 1 -- 172.21.15.98:0/813215035 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400b820 con 0x7fb1f406ed20 2023-10-06T01:41:43.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1f2ffd700 1 -- 172.21.15.98:0/813215035 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e400b980 con 0x7fb1f406ed20 2023-10-06T01:41:43.663 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1f2ffd700 1 -- 172.21.15.98:0/813215035 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400bd20 con 0x7fb1f406ed20 2023-10-06T01:41:43.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 -- 172.21.15.98:0/813215035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f406ed20 msgr2=0x7fb1f40b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f406ed20 0x7fb1f40b23a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099b0 tx=0x7fb1e4009250 comp rx=0 tx=0).stop 2023-10-06T01:41:43.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 -- 172.21.15.98:0/813215035 shutdown_connections 2023-10-06T01:41:43.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f40aee20 0x7fb1f40af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.664 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 --2- 172.21.15.98:0/813215035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f406ed20 0x7fb1f40b23a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 -- 172.21.15.98:0/813215035 >> 172.21.15.98:0/813215035 conn(0x7fb1f4068330 msgr2=0x7fb1f4068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 -- 172.21.15.98:0/813215035 shutdown_connections 2023-10-06T01:41:43.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.651+0000 7fb1fa922700 1 -- 172.21.15.98:0/813215035 wait complete. 2023-10-06T01:41:43.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.652+0000 7fb1fa922700 1 Processor -- start 2023-10-06T01:41:43.665 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 -- start start 2023-10-06T01:41:43.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f406ed20 0x7fb1f40ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1f406f0f0 0x7fb1f40ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 0x7fb1f41694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.666 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40abf00 con 0x7fb1f40aee20 2023-10-06T01:41:43.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40ae180 con 0x7fb1f406f0f0 2023-10-06T01:41:43.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1fa922700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40ae460 con 0x7fb1f406ed20 2023-10-06T01:41:43.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1f8ebf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 0x7fb1f41694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.667 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1f8ebf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 0x7fb1f41694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51082/0 (socket says 172.21.15.98:51082) 2023-10-06T01:41:43.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1f8ebf700 1 -- 172.21.15.98:0/527814909 learned_addr learned my addr 172.21.15.98:0/527814909 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1f3fff700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f406ed20 0x7fb1f40ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.653+0000 7fb1f37fe700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1f406f0f0 0x7fb1f40ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.668 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 -- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f406ed20 msgr2=0x7fb1f40ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1f406ed20 0x7fb1f40ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 -- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1f406f0f0 msgr2=0x7fb1f40ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1f406f0f0 0x7fb1f40ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40094f0 con 0x7fb1f40aee20 2023-10-06T01:41:43.669 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1f8ebf700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 0x7fb1f41694f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00ceb0 tx=0x7fb1ec007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.670 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00ad90 con 0x7fb1f40aee20 2023-10-06T01:41:43.670 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec016430 con 0x7fb1f40aee20 2023-10-06T01:41:43.670 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f40ae740 con 0x7fb1f40aee20 2023-10-06T01:41:43.670 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0167d0 con 0x7fb1f40aee20 2023-10-06T01:41:43.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.654+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f40ade10 con 0x7fb1f40aee20 2023-10-06T01:41:43.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.655+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1f40a8b30 con 0x7fb1f40aee20 2023-10-06T01:41:43.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.655+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1ec003020 con 0x7fb1f40aee20 2023-10-06T01:41:43.671 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1dbfff700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb1e0066f30 0x7fb1e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fb1ec012070 con 0x7fb1f40aee20 2023-10-06T01:41:43.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1f3fff700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb1e0066f30 0x7fb1e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1dbfff700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fb1e006d8f0 0x7fb1e006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.672 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1e00703f0 con 0x7fb1e006d8f0 2023-10-06T01:41:43.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1ec0877a0 con 0x7fb1f40aee20 2023-10-06T01:41:43.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1f37fe700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fb1e006d8f0 0x7fb1e006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1f3fff700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb1e0066f30 0x7fb1e00693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099b0 tx=0x7fb1e4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.656+0000 7fb1f37fe700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fb1e006d8f0 0x7fb1e006fd20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.673 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.657+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== osd.2 v2:172.21.15.98:6824/120679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb1e00703f0 con 0x7fb1e006d8f0 2023-10-06T01:41:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.670+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 --> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb1f40cd260 con 0x7fb1e006d8f0 2023-10-06T01:41:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1dbfff700 1 -- 172.21.15.98:0/527814909 <== osd.2 v2:172.21.15.98:6824/120679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb1f40cd260 con 0x7fb1e006d8f0 2023-10-06T01:41:43.675 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:43.676 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:43.676 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:43.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fb1e006d8f0 msgr2=0x7fb1e006fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6824/120679,v1:172.21.15.98:6825/120679] conn(0x7fb1e006d8f0 0x7fb1e006fd20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb1e0066f30 msgr2=0x7fb1e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.678 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fb1e0066f30 0x7fb1e00693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099b0 tx=0x7fb1e4016000 comp rx=0 tx=0).stop 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 msgr2=0x7fb1f41694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 --2- 172.21.15.98:0/527814909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f40aee20 0x7fb1f41694f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00ceb0 tx=0x7fb1ec007870 comp rx=0 tx=0).stop 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1f8ebf700 1 -- 172.21.15.98:0/527814909 reap_dead start 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 shutdown_connections 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 >> 172.21.15.98:0/527814909 conn(0x7fb1f4068330 msgr2=0x7fb1f4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 shutdown_connections 2023-10-06T01:41:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.671+0000 7fb1fa922700 1 -- 172.21.15.98:0/527814909 wait complete. 2023-10-06T01:41:43.683 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T01:41:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.797+0000 7f9d59551700 1 Processor -- start 2023-10-06T01:41:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.797+0000 7f9d59551700 1 -- start start 2023-10-06T01:41:43.800 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d59551700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 0x7f9d541582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d59551700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d540adba0 0x7f9d540ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d59551700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d54157e20 con 0x7f9d540ce710 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d59551700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540cbbd0 con 0x7f9d540adba0 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d59551700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540ceb30 con 0x7f9d540ca3b0 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d4a7fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d540adba0 0x7f9d540ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d537fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9d540ce710 0x7f9d5415af10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42476/0 (socket says 172.21.15.98:42476) 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d4a7fc700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d540adba0 0x7f9d540ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51098/0 (socket says 172.21.15.98:51098) 2023-10-06T01:41:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.798+0000 7f9d537fe700 1 -- 172.21.15.98:0/1060151278 learned_addr learned my addr 172.21.15.98:0/1060151278 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d52ffd700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 0x7f9d541582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/1060151278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 msgr2=0x7f9d541582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d4a7fc700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 0x7f9d541582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/1060151278 >> v1:172.21.15.98:6789/0 conn(0x7f9d540ce710 legacy=0x7f9d5415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/1060151278 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540cb610 con 0x7f9d540adba0 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d52ffd700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 0x7f9d541582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d527fc700 1 -- 172.21.15.98:0/1060151278 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916833014 0 0) 0x7f9d54157e20 con 0x7f9d540ce710 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d4a7fc700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540adba0 0x7f9d540ce1d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098e0 tx=0x7f9d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b64ef1491b7bda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d527fc700 1 -- 172.21.15.98:0/1060151278 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c009e80 con 0x7f9d540adba0 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d527fc700 1 -- 172.21.15.98:0/1060151278 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00b860 con 0x7f9d540adba0 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d527fc700 1 -- 172.21.15.98:0/1060151278 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00bbe0 con 0x7f9d540adba0 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 -- 172.21.15.98:0/1060151278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540adba0 msgr2=0x7f9d540ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540adba0 0x7f9d540ce1d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098e0 tx=0x7f9d3c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 -- 172.21.15.98:0/1060151278 shutdown_connections 2023-10-06T01:41:43.803 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ca3b0 0x7f9d541582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 --2- 172.21.15.98:0/1060151278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540adba0 0x7f9d540ce1d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 -- 172.21.15.98:0/1060151278 >> 172.21.15.98:0/1060151278 conn(0x7f9d54070810 msgr2=0x7f9d54070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 -- 172.21.15.98:0/1060151278 shutdown_connections 2023-10-06T01:41:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.799+0000 7f9d59551700 1 -- 172.21.15.98:0/1060151278 wait complete. 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 Processor -- start 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 -- start start 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d540adba0 0x7f9d540afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ce710 0x7f9d54152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c9d80 con 0x7f9d540ca3b0 2023-10-06T01:41:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540b1260 con 0x7f9d540adba0 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d59551700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540b1500 con 0x7f9d540ce710 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.801+0000 7f9d4a7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51106/0 (socket says 172.21.15.98:51106) 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/3076513045 learned_addr learned my addr 172.21.15.98:0/3076513045 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ce710 msgr2=0x7f9d54152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ce710 0x7f9d54152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d540adba0 msgr2=0x7f9d540afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d540adba0 0x7f9d540afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 -- 172.21.15.98:0/3076513045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009420 con 0x7f9d540ca3b0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4a7fc700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9d540cdfe0 tx=0x7f9d3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00b9d0 con 0x7f9d540ca3b0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00bb50 con 0x7f9d540ca3b0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540b17e0 con 0x7f9d540ca3b0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c025670 con 0x7f9d540ca3b0 2023-10-06T01:41:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.802+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540b1da0 con 0x7f9d540ca3b0 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.803+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d540cd610 con 0x7f9d540ca3b0 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.803+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d3c023070 con 0x7f9d540ca3b0 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.804+0000 7f9d4b7fe700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 0x7f9d340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.804+0000 7f9d52ffd700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 0x7f9d340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.804+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f9d3c085c20 con 0x7f9d540ca3b0 2023-10-06T01:41:43.810 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.804+0000 7f9d52ffd700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 0x7f9d340693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d44007aa0 tx=0x7f9d44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:43.811 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.808+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d3c052e90 con 0x7f9d540ca3b0 2023-10-06T01:41:43.991 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.990+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d540b1f90 con 0x7f9d34066f30 2023-10-06T01:41:43.991 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.990+0000 7f9d4b7fe700 1 -- 172.21.15.98:0/3076513045 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9d540b1f90 con 0x7f9d34066f30 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 msgr2=0x7f9d340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 0x7f9d340693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d44007aa0 tx=0x7f9d44008040 comp rx=0 tx=0).stop 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 msgr2=0x7f9d540c1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9d540cdfe0 tx=0x7f9d3c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 shutdown_connections 2023-10-06T01:41:43.993 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f9d34066f30 0x7f9d340693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d540ce710 0x7f9d54152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d540adba0 0x7f9d540afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 --2- 172.21.15.98:0/3076513045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d540ca3b0 0x7f9d540c1d70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 >> 172.21.15.98:0/3076513045 conn(0x7f9d54070810 msgr2=0x7f9d541554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 shutdown_connections 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:43.992+0000 7f9d59551700 1 -- 172.21.15.98:0/3076513045 wait complete. 2023-10-06T01:41:43.994 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T01:41:44.005 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T01:41:44.005 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:44.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.121+0000 7f36992d1700 1 Processor -- start 2023-10-06T01:41:44.122 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.121+0000 7f36992d1700 1 -- start start 2023-10-06T01:41:44.124 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36992d1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 0x7f36940aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36992d1700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36940b2d20 con 0x7f36940b23a0 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36992d1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36940b0450 con 0x7f369406ed20 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36992d1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369406fb80 con 0x7f369415a780 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36927fc700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f36940b23a0 0x7f36940b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42488/0 (socket says 172.21.15.98:42488) 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36927fc700 1 -- 172.21.15.98:0/3395156035 learned_addr learned my addr 172.21.15.98:0/3395156035 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.122+0000 7f36937fe700 1 --2- 172.21.15.98:0/3395156035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 0x7f36940aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137305368 0 0) 0x7f36940b2d20 con 0x7f36940b23a0 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f367c003570 con 0x7f36940b23a0 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424570054 0 0) 0x7f36940b0450 con 0x7f369406ed20 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36940b2d20 con 0x7f369406ed20 2023-10-06T01:41:44.125 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f36937fe700 1 -- 172.21.15.98:0/3395156035 >> v1:172.21.15.184:6789/0 conn(0x7f369406ed20 legacy=0x7f369406f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f36937fe700 1 -- 172.21.15.98:0/3395156035 >> v1:172.21.15.98:6789/0 conn(0x7f36940b23a0 legacy=0x7f36940b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f36937fe700 1 -- 172.21.15.98:0/3395156035 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940b2a40 con 0x7f369415a780 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f36937fe700 1 --2- 172.21.15.98:0/3395156035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 0x7f36940aee20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f368c009910 tx=0x7f368c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37f1275e92db797f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c011630 con 0x7f369415a780 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c011790 con 0x7f369415a780 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.123+0000 7f3691ffb700 1 -- 172.21.15.98:0/3395156035 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c011b30 con 0x7f369415a780 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 -- 172.21.15.98:0/3395156035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 msgr2=0x7f36940aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 --2- 172.21.15.98:0/3395156035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 0x7f36940aee20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f368c009910 tx=0x7f368c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 -- 172.21.15.98:0/3395156035 shutdown_connections 2023-10-06T01:41:44.126 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 --2- 172.21.15.98:0/3395156035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f369415a780 0x7f36940aee20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 -- 172.21.15.98:0/3395156035 >> 172.21.15.98:0/3395156035 conn(0x7f3694068330 msgr2=0x7f3694068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 -- 172.21.15.98:0/3395156035 shutdown_connections 2023-10-06T01:41:44.127 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.124+0000 7f36992d1700 1 -- 172.21.15.98:0/3395156035 wait complete. 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.125+0000 7f36992d1700 1 Processor -- start 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.125+0000 7f36992d1700 1 -- start start 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369406ed20 0x7f36940cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36940b23a0 0x7f3694158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 0x7f3694159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940ce9d0 con 0x7f369415a780 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369415a530 con 0x7f369406ed20 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940ce0a0 con 0x7f36940b23a0 2023-10-06T01:41:44.131 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 0x7f3694159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 0x7f3694159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51120/0 (socket says 172.21.15.98:51120) 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 -- 172.21.15.98:0/2994680284 learned_addr learned my addr 172.21.15.98:0/2994680284 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f3692ffd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369406ed20 0x7f36940cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36927fc700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36940b23a0 0x7f3694158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 -- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36940b23a0 msgr2=0x7f3694158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36940b23a0 0x7f3694158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 -- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369406ed20 msgr2=0x7f36940cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369406ed20 0x7f36940cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c009420 con 0x7f369415a780 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f3692ffd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f369406ed20 0x7f36940cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:44.132 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36937fe700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 0x7f3694159a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f36940893c0 tx=0x7f368c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c01f070 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c002f70 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36940ce380 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c011b30 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.126+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369416a770 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.127+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3694064ad0 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f368c023070 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3680066f30 0x7f36800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f368c085a80 con 0x7f369415a780 2023-10-06T01:41:44.133 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f3692ffd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3680066f30 0x7f36800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f368006d8f0 0x7f368006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36800703f0 con 0x7f368006d8f0 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f368c087ea0 con 0x7f369415a780 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f36927fc700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f368006d8f0 0x7f368006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.128+0000 7f3692ffd700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3680066f30 0x7f36800693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3684002f20 tx=0x7f3684009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.129+0000 7f36927fc700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f368006d8f0 0x7f368006fd20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.134 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.129+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== osd.3 v2:172.21.15.98:6809/120655 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f36800703f0 con 0x7f368006d8f0 2023-10-06T01:41:44.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 --> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f369404bb80 con 0x7f368006d8f0 2023-10-06T01:41:44.143 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f367affd700 1 -- 172.21.15.98:0/2994680284 <== osd.3 v2:172.21.15.98:6809/120655 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f369404bb80 con 0x7f368006d8f0 2023-10-06T01:41:44.144 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:44.144 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:44.144 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f368006d8f0 msgr2=0x7f368006fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6809/120655,v1:172.21.15.98:6811/120655] conn(0x7f368006d8f0 0x7f368006fd20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3680066f30 msgr2=0x7f36800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f3680066f30 0x7f36800693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3684002f20 tx=0x7f3684009040 comp rx=0 tx=0).stop 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 msgr2=0x7f3694159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 --2- 172.21.15.98:0/2994680284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369415a780 0x7f3694159a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f36940893c0 tx=0x7f368c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36937fe700 1 -- 172.21.15.98:0/2994680284 reap_dead start 2023-10-06T01:41:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 shutdown_connections 2023-10-06T01:41:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 >> 172.21.15.98:0/2994680284 conn(0x7f3694068330 msgr2=0x7f3694156990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 shutdown_connections 2023-10-06T01:41:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.143+0000 7f36992d1700 1 -- 172.21.15.98:0/2994680284 wait complete. 2023-10-06T01:41:44.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T01:41:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.271+0000 7fa42fb83700 1 Processor -- start 2023-10-06T01:41:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.271+0000 7fa42fb83700 1 -- start start 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.271+0000 7fa42fb83700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa4281595a0 0x7fa4280aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42fb83700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa4280b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42fb83700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280b0ca0 con 0x7fa4280b49e0 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42fb83700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280b0f80 con 0x7fa4281595a0 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42fb83700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280b1260 con 0x7fa4280b07a0 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa4281595a0 0x7fa4280aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d11e700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa4280b49e0 0x7fa4280aaab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42502/0 (socket says 172.21.15.98:42502) 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d11e700 1 -- 172.21.15.98:0/702799507 learned_addr learned my addr 172.21.15.98:0/702799507 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 -- 172.21.15.98:0/702799507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 msgr2=0x7fa4280b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 --2- 172.21.15.98:0/702799507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa4280b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 -- 172.21.15.98:0/702799507 >> v1:172.21.15.98:6789/0 conn(0x7fa4280b49e0 legacy=0x7fa4280aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 -- 172.21.15.98:0/702799507 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280b1540 con 0x7fa4281595a0 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42c91d700 1 -- 172.21.15.98:0/702799507 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340339512 0 0) 0x7fa4280b0ca0 con 0x7fa4280b49e0 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42d91f700 1 --2- 172.21.15.98:0/702799507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4281595a0 0x7fa4280aa570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa424009900 tx=0x7fa424009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a4dc9eb28e8376 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42c91d700 1 -- 172.21.15.98:0/702799507 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424009ea0 con 0x7fa4281595a0 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42c91d700 1 -- 172.21.15.98:0/702799507 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424005c10 con 0x7fa4281595a0 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.272+0000 7fa42c91d700 1 -- 172.21.15.98:0/702799507 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400b610 con 0x7fa4281595a0 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 -- 172.21.15.98:0/702799507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4281595a0 msgr2=0x7fa4280aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 --2- 172.21.15.98:0/702799507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4281595a0 0x7fa4280aa570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa424009900 tx=0x7fa424009120 comp rx=0 tx=0).stop 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 -- 172.21.15.98:0/702799507 shutdown_connections 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 --2- 172.21.15.98:0/702799507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa4280b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 --2- 172.21.15.98:0/702799507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4281595a0 0x7fa4280aa570 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 -- 172.21.15.98:0/702799507 >> 172.21.15.98:0/702799507 conn(0x7fa428099830 msgr2=0x7fa42806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 -- 172.21.15.98:0/702799507 shutdown_connections 2023-10-06T01:41:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.273+0000 7fa42fb83700 1 -- 172.21.15.98:0/702799507 wait complete. 2023-10-06T01:41:44.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.274+0000 7fa42fb83700 1 Processor -- start 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.274+0000 7fa42fb83700 1 -- start start 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa42815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4281595a0 0x7fa428158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815a8f0 con 0x7fa4280b49e0 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428169be0 con 0x7fa4281595a0 2023-10-06T01:41:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815b3a0 con 0x7fa4280b07a0 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51132/0 (socket says 172.21.15.98:51132) 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 -- 172.21.15.98:0/2463633827 learned_addr learned my addr 172.21.15.98:0/2463633827 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 -- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 msgr2=0x7fa42815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa42815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 -- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4281595a0 msgr2=0x7fa428158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.281 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4281595a0 0x7fa428158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 -- 172.21.15.98:0/2463633827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4240093c0 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42d11e700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c7e0 tx=0x7fa418007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418005c00 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418005d60 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42815ab00 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.275+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180072e0 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.276+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42815b0c0 con 0x7fa4280b49e0 2023-10-06T01:41:44.282 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.276+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa428064ad0 con 0x7fa4280b49e0 2023-10-06T01:41:44.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.277+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa41801b3b0 con 0x7fa4280b49e0 2023-10-06T01:41:44.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.277+0000 7fa41dffb700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 0x7fa40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.277+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fa4180839a0 con 0x7fa4280b49e0 2023-10-06T01:41:44.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.277+0000 7fa42d91f700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 0x7fa40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.283 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.277+0000 7fa42d91f700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 0x7fa40c0693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa424009900 tx=0x7fa424016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.284 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.281+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa41804b5a0 con 0x7fa4280b49e0 2023-10-06T01:41:44.466 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.466+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa428169d20 con 0x7fa40c066f30 2023-10-06T01:41:44.467 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.466+0000 7fa41dffb700 1 -- 172.21.15.98:0/2463633827 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa428169d20 con 0x7fa40c066f30 2023-10-06T01:41:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 msgr2=0x7fa40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 0x7fa40c0693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa424009900 tx=0x7fa424016000 comp rx=0 tx=0).stop 2023-10-06T01:41:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 msgr2=0x7fa42815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c7e0 tx=0x7fa418007d90 comp rx=0 tx=0).stop 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 shutdown_connections 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa40c066f30 0x7fa40c0693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4281595a0 0x7fa428158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4280b07a0 0x7fa42815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 --2- 172.21.15.98:0/2463633827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4280b49e0 0x7fa42815a3b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 >> 172.21.15.98:0/2463633827 conn(0x7fa428099830 msgr2=0x7fa428068960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 shutdown_connections 2023-10-06T01:41:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.468+0000 7fa42fb83700 1 -- 172.21.15.98:0/2463633827 wait complete. 2023-10-06T01:41:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T01:41:44.480 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T01:41:44.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:44.598 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.598+0000 7efdc6253700 1 Processor -- start 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.598+0000 7efdc6253700 1 -- start start 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdc6253700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 0x7efdc00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdc6253700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 0x7efdc00af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdc6253700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc00b2d20 con 0x7efdc006ed20 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdc6253700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc006fb80 con 0x7efdc00b23a0 2023-10-06T01:41:44.599 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdc6253700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00b2840 con 0x7efdc00aee20 2023-10-06T01:41:44.601 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbeffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 0x7efdc00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 0x7efdc00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbf7fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7efdc006ed20 0x7efdc006f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:51746/0 (socket says 172.21.15.98:51746) 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbf7fe700 1 -- 172.21.15.98:0/1889422198 learned_addr learned my addr 172.21.15.98:0/1889422198 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbffff700 1 -- 172.21.15.98:0/1889422198 >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 msgr2=0x7efdc00b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbffff700 1 --2- 172.21.15.98:0/1889422198 >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 0x7efdc00b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbffff700 1 -- 172.21.15.98:0/1889422198 >> v1:172.21.15.184:6789/0 conn(0x7efdc006ed20 legacy=0x7efdc006f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.602 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbffff700 1 -- 172.21.15.98:0/1889422198 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00b0450 con 0x7efdc00aee20 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbe7fc700 1 -- 172.21.15.98:0/1889422198 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467364991 0 0) 0x7efdc00b2d20 con 0x7efdc006ed20 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.599+0000 7efdbeffd700 1 --2- 172.21.15.98:0/1889422198 >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 0x7efdc00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdbffff700 1 --2- 172.21.15.98:0/1889422198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 0x7efdc00af280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e60 tx=0x7efdb8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3544d53b9f03cfa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdbe7fc700 1 -- 172.21.15.98:0/1889422198 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800e070 con 0x7efdc00aee20 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdbe7fc700 1 -- 172.21.15.98:0/1889422198 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb800bb60 con 0x7efdc00aee20 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdbe7fc700 1 -- 172.21.15.98:0/1889422198 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8012430 con 0x7efdc00aee20 2023-10-06T01:41:44.603 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 -- 172.21.15.98:0/1889422198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 msgr2=0x7efdc00af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 --2- 172.21.15.98:0/1889422198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 0x7efdc00af280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e60 tx=0x7efdb8009780 comp rx=0 tx=0).stop 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 -- 172.21.15.98:0/1889422198 shutdown_connections 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 --2- 172.21.15.98:0/1889422198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00aee20 0x7efdc00af280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 --2- 172.21.15.98:0/1889422198 >> v2:172.21.15.184:3300/0 conn(0x7efdc00b23a0 0x7efdc00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 -- 172.21.15.98:0/1889422198 >> 172.21.15.98:0/1889422198 conn(0x7efdc0068330 msgr2=0x7efdc0068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 -- 172.21.15.98:0/1889422198 shutdown_connections 2023-10-06T01:41:44.604 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.600+0000 7efdc6253700 1 -- 172.21.15.98:0/1889422198 wait complete. 2023-10-06T01:41:44.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 Processor -- start 2023-10-06T01:41:44.607 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 -- start start 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 0x7efdc015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00aee20 0x7efdc00c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00b23a0 0x7efdc00c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc0158620 con 0x7efdc006ed20 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc01596e0 con 0x7efdc00aee20 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdc6253700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc0158db0 con 0x7efdc00b23a0 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdbf7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 0x7efdc015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdbf7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 0x7efdc015bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51148/0 (socket says 172.21.15.98:51148) 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.602+0000 7efdbf7fe700 1 -- 172.21.15.98:0/2438224790 learned_addr learned my addr 172.21.15.98:0/2438224790 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbffff700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00b23a0 0x7efdc00c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbeffd700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00aee20 0x7efdc00c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 -- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00b23a0 msgr2=0x7efdc00c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc00b23a0 0x7efdc00c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 -- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00aee20 msgr2=0x7efdc00c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00aee20 0x7efdc00c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009420 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbeffd700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00aee20 0x7efdc00c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdbf7fe700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 0x7efdc015bc50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c5b0 tx=0x7efdb0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0005b30 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb0005c90 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb8009a20 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0007320 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.603+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc0159370 con 0x7efdc006ed20 2023-10-06T01:41:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.604+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efdc0064ad0 con 0x7efdc006ed20 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.604+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdb001c440 con 0x7efdc006ed20 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efda77fe700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efdac066f30 0x7efdac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efdbeffd700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efdac066f30 0x7efdac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7efdb0083950 con 0x7efdc006ed20 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efda77fe700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7efdac06d8f0 0x7efdac06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdac0703f0 con 0x7efdac06d8f0 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdb0088240 con 0x7efdc006ed20 2023-10-06T01:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efdbeffd700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efdac066f30 0x7efdac0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efdb40093f0 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.605+0000 7efdbffff700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7efdac06d8f0 0x7efdac06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.606+0000 7efdbffff700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7efdac06d8f0 0x7efdac06fd20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.612 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.606+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== osd.4 v2:172.21.15.184:6824/120887 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7efdac0703f0 con 0x7efdac06d8f0 2023-10-06T01:41:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.620+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 --> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efdc0159500 con 0x7efdac06d8f0 2023-10-06T01:41:44.621 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.620+0000 7efda77fe700 1 -- 172.21.15.98:0/2438224790 <== osd.4 v2:172.21.15.184:6824/120887 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efdc0159500 con 0x7efdac06d8f0 2023-10-06T01:41:44.621 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:44.621 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:44.621 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7efdac06d8f0 msgr2=0x7efdac06fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.184:6824/120887,v1:172.21.15.184:6825/120887] conn(0x7efdac06d8f0 0x7efdac06fd20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efdac066f30 msgr2=0x7efdac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.622 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7efdac066f30 0x7efdac0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efdb40093f0 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 msgr2=0x7efdc015bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 --2- 172.21.15.98:0/2438224790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdc006ed20 0x7efdc015bc50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c5b0 tx=0x7efdb0001070 comp rx=0 tx=0).stop 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdbffff700 1 -- 172.21.15.98:0/2438224790 reap_dead start 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 shutdown_connections 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 >> 172.21.15.98:0/2438224790 conn(0x7efdc0068330 msgr2=0x7efdc0156980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 shutdown_connections 2023-10-06T01:41:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.621+0000 7efdc6253700 1 -- 172.21.15.98:0/2438224790 wait complete. 2023-10-06T01:41:44.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T01:41:44.748 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.748+0000 7f62c359e700 1 Processor -- start 2023-10-06T01:41:44.749 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.748+0000 7f62c359e700 1 -- start start 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62c4159480 0x7f62c40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c40b3610 0x7f62c40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f62c40cc1b0 0x7f62c40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c40cb7e0 con 0x7f62c40cc1b0 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c40b08a0 con 0x7f62c4159480 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40cb190 con 0x7f62c40b3610 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f62c40cc1b0 0x7f62c40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f62c40cc1b0 0x7f62c40b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51152/0 (socket says 172.21.15.98:51152) 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 -- 172.21.15.98:0/2533347208 learned_addr learned my addr 172.21.15.98:0/2533347208 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c259c700 1 --2- 172.21.15.98:0/2533347208 >> v2:172.21.15.184:3300/0 conn(0x7f62c4159480 0x7f62c40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c1d9b700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c40b3610 0x7f62c40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 -- 172.21.15.98:0/2533347208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c40b3610 msgr2=0x7f62c40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c40b3610 0x7f62c40cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 -- 172.21.15.98:0/2533347208 >> v2:172.21.15.184:3300/0 conn(0x7f62c4159480 msgr2=0x7f62c40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 --2- 172.21.15.98:0/2533347208 >> v2:172.21.15.184:3300/0 conn(0x7f62c4159480 0x7f62c40b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 -- 172.21.15.98:0/2533347208 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c40cb470 con 0x7f62c40cc1b0 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.749+0000 7f62c2d9d700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40cc1b0 0x7f62c40b2140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f62c406fa30 tx=0x7f62bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33db732cc9600a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c159a700 1 -- 172.21.15.98:0/2533347208 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00e070 con 0x7f62c40cc1b0 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c159a700 1 -- 172.21.15.98:0/2533347208 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc00b970 con 0x7f62c40cc1b0 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c159a700 1 -- 172.21.15.98:0/2533347208 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0092e0 con 0x7f62c40cc1b0 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 -- 172.21.15.98:0/2533347208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40cc1b0 msgr2=0x7f62c40b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40cc1b0 0x7f62c40b2140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f62c406fa30 tx=0x7f62bc009840 comp rx=0 tx=0).stop 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 -- 172.21.15.98:0/2533347208 shutdown_connections 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 --2- 172.21.15.98:0/2533347208 >> v2:172.21.15.184:3300/0 conn(0x7f62c4159480 0x7f62c40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40cc1b0 0x7f62c40b2140 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 --2- 172.21.15.98:0/2533347208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c40b3610 0x7f62c40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 -- 172.21.15.98:0/2533347208 >> 172.21.15.98:0/2533347208 conn(0x7f62c406ed20 msgr2=0x7f62c406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 -- 172.21.15.98:0/2533347208 shutdown_connections 2023-10-06T01:41:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.750+0000 7f62c359e700 1 -- 172.21.15.98:0/2533347208 wait complete. 2023-10-06T01:41:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.751+0000 7f62c359e700 1 Processor -- start 2023-10-06T01:41:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.751+0000 7f62c359e700 1 -- start start 2023-10-06T01:41:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c40cc1b0 0x7f62c40c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 0x7f62c40c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40cf7e0 con 0x7f62c40b3610 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c415a000 con 0x7f62c40cc1b0 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40cf220 con 0x7f62c4159480 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51158/0 (socket says 172.21.15.98:51158) 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 -- 172.21.15.98:0/2144580309 learned_addr learned my addr 172.21.15.98:0/2144580309 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c1d9b700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c40cc1b0 0x7f62c40c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c2d9d700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 0x7f62c40c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 -- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 msgr2=0x7f62c40c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 0x7f62c40c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 -- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c40cc1b0 msgr2=0x7f62c40c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c40cc1b0 0x7f62c40c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c259c700 1 -- 172.21.15.98:0/2144580309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009ae0 con 0x7f62c40b3610 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.752+0000 7f62c2d9d700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 0x7f62c40c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62c259c700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e930 tx=0x7f62b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400eed0 con 0x7f62c40b3610 2023-10-06T01:41:44.758 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400c850 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c40cf500 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400cbf0 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.753+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c40d0a60 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.754+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c405f270 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.754+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62b4013470 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.755+0000 7f62b27fc700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 0x7f62a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.755+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f62b4084ea0 con 0x7f62c40b3610 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.755+0000 7f62c1d9b700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 0x7f62a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.755+0000 7f62c1d9b700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 0x7f62a40693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62c40d0080 tx=0x7f62b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.758+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62b4052190 con 0x7f62c40b3610 2023-10-06T01:41:44.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.942+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f62c4159850 con 0x7f62a4066f30 2023-10-06T01:41:44.943 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.942+0000 7f62b27fc700 1 -- 172.21.15.98:0/2144580309 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f62c4159850 con 0x7f62a4066f30 2023-10-06T01:41:44.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 msgr2=0x7f62a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 0x7f62a40693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62c40d0080 tx=0x7f62b8008040 comp rx=0 tx=0).stop 2023-10-06T01:41:44.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 msgr2=0x7f62c40ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:44.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e930 tx=0x7f62b40091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.945 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 shutdown_connections 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f62a4066f30 0x7f62a40693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62c4159480 0x7f62c40c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62c40b3610 0x7f62c40ce840 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 --2- 172.21.15.98:0/2144580309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c40cc1b0 0x7f62c40c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 >> 172.21.15.98:0/2144580309 conn(0x7f62c406ed20 msgr2=0x7f62c4155010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 shutdown_connections 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:44.944+0000 7f62c359e700 1 -- 172.21.15.98:0/2144580309 wait complete. 2023-10-06T01:41:44.946 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T01:41:44.956 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T01:41:44.957 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.073+0000 7f14bf50e700 1 Processor -- start 2023-10-06T01:41:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 -- start start 2023-10-06T01:41:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14b806ed20 0x7f14b80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14b80aee20 0x7f14b80af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80b2d20 con 0x7f14b806f0f0 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b806fb80 con 0x7f14b80aee20 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.074+0000 7f14bf50e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80b0450 con 0x7f14b806ed20 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14b80aee20 0x7f14b80af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bcaa9700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14b806f0f0 0x7f14b80b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42516/0 (socket says 172.21.15.98:42516) 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bcaa9700 1 -- 172.21.15.98:0/541687874 learned_addr learned my addr 172.21.15.98:0/541687874 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bd2aa700 1 --2- 172.21.15.98:0/541687874 >> v2:172.21.15.184:3300/0 conn(0x7f14b806ed20 0x7f14b80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 -- 172.21.15.98:0/541687874 >> v2:172.21.15.184:3300/0 conn(0x7f14b806ed20 msgr2=0x7f14b80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 --2- 172.21.15.98:0/541687874 >> v2:172.21.15.184:3300/0 conn(0x7f14b806ed20 0x7f14b80b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 -- 172.21.15.98:0/541687874 >> v1:172.21.15.98:6789/0 conn(0x7f14b806f0f0 legacy=0x7f14b80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 -- 172.21.15.98:0/541687874 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80af7c0 con 0x7f14b80aee20 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14b7fff700 1 -- 172.21.15.98:0/541687874 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184822219 0 0) 0x7f14b80b2d20 con 0x7f14b806f0f0 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14bdaab700 1 --2- 172.21.15.98:0/541687874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80aee20 0x7f14b80af280 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14b80cdd30 tx=0x7f14b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93da9ec09c70c859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14b7fff700 1 -- 172.21.15.98:0/541687874 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b000e070 con 0x7f14b80aee20 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14b7fff700 1 -- 172.21.15.98:0/541687874 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b000bb60 con 0x7f14b80aee20 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.075+0000 7f14b7fff700 1 -- 172.21.15.98:0/541687874 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0012430 con 0x7f14b80aee20 2023-10-06T01:41:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 -- 172.21.15.98:0/541687874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80aee20 msgr2=0x7f14b80af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 --2- 172.21.15.98:0/541687874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80aee20 0x7f14b80af280 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14b80cdd30 tx=0x7f14b0009780 comp rx=0 tx=0).stop 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 -- 172.21.15.98:0/541687874 shutdown_connections 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 --2- 172.21.15.98:0/541687874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80aee20 0x7f14b80af280 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 --2- 172.21.15.98:0/541687874 >> v2:172.21.15.184:3300/0 conn(0x7f14b806ed20 0x7f14b80b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 -- 172.21.15.98:0/541687874 >> 172.21.15.98:0/541687874 conn(0x7f14b8068330 msgr2=0x7f14b8068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 -- 172.21.15.98:0/541687874 shutdown_connections 2023-10-06T01:41:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.076+0000 7f14bf50e700 1 -- 172.21.15.98:0/541687874 wait complete. 2023-10-06T01:41:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.077+0000 7f14bf50e700 1 Processor -- start 2023-10-06T01:41:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.077+0000 7f14bf50e700 1 -- start start 2023-10-06T01:41:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 0x7f14b8158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b806f0f0 0x7f14b80cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b80aee20 0x7f14b80d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8158870 con 0x7f14b806ed20 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c7ea0 con 0x7f14b80aee20 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bf50e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c8930 con 0x7f14b806f0f0 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 0x7f14b8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 0x7f14b8158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51166/0 (socket says 172.21.15.98:51166) 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 -- 172.21.15.98:0/111029025 learned_addr learned my addr 172.21.15.98:0/111029025 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bdaab700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b80aee20 0x7f14b80d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bcaa9700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b806f0f0 0x7f14b80cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 -- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b806f0f0 msgr2=0x7f14b80cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b806f0f0 0x7f14b80cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 -- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b80aee20 msgr2=0x7f14b80d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b80aee20 0x7f14b80d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0009a20 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bdaab700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b80aee20 0x7f14b80d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14bd2aa700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 0x7f14b8158040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e930 tx=0x7f14a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.078+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800eed0 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.079+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800c850 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.079+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b80c8090 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.079+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800cbf0 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.079+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b80c8650 con 0x7f14b806ed20 2023-10-06T01:41:45.084 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.079+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14b80c8c10 con 0x7f14b806ed20 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14a8013470 con 0x7f14b806ed20 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f14a4066f30 0x7f14a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f14a8084770 con 0x7f14b806ed20 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14bcaa9700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f14a4066f30 0x7f14a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f14a406d780 0x7f14a406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14a4070280 con 0x7f14a406d780 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.080+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a8086db0 con 0x7f14b806ed20 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.081+0000 7f14bdaab700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f14a406d780 0x7f14a406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.081+0000 7f14bcaa9700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f14a4066f30 0x7f14a40693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b50a0 tx=0x7f14ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.081+0000 7f14bdaab700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f14a406d780 0x7f14a406fbb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.082+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== osd.5 v2:172.21.15.184:6802/120874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f14a4070280 con 0x7f14a406d780 2023-10-06T01:41:45.095 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.095+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 --> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f14b8064ad0 con 0x7f14a406d780 2023-10-06T01:41:45.096 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.095+0000 7f14b57fa700 1 -- 172.21.15.98:0/111029025 <== osd.5 v2:172.21.15.184:6802/120874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f14b8064ad0 con 0x7f14a406d780 2023-10-06T01:41:45.096 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:45.096 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:45.096 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f14a406d780 msgr2=0x7f14a406fbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.184:6802/120874,v1:172.21.15.184:6804/120874] conn(0x7f14a406d780 0x7f14a406fbb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f14a4066f30 msgr2=0x7f14a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f14a4066f30 0x7f14a40693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b50a0 tx=0x7f14ac009040 comp rx=0 tx=0).stop 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 msgr2=0x7f14b8158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 --2- 172.21.15.98:0/111029025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b806ed20 0x7f14b8158040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e930 tx=0x7f14a80091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.098 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bdaab700 1 -- 172.21.15.98:0/111029025 reap_dead start 2023-10-06T01:41:45.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 shutdown_connections 2023-10-06T01:41:45.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 >> 172.21.15.98:0/111029025 conn(0x7f14b8068330 msgr2=0x7f14b81567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 shutdown_connections 2023-10-06T01:41:45.099 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.096+0000 7f14bf50e700 1 -- 172.21.15.98:0/111029025 wait complete. 2023-10-06T01:41:45.108 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T01:41:45.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 Processor -- start 2023-10-06T01:41:45.223 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 -- start start 2023-10-06T01:41:45.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec241582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fec240ce8e0 0x7fec2415af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec24157e20 con 0x7fec240adba0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240cbbd0 con 0x7fec240ce8e0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec2b07b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240cb610 con 0x7fec240ca3b0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.223+0000 7fec23fff700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fec240adba0 0x7fec240ce1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42528/0 (socket says 172.21.15.98:42528) 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec23fff700 1 -- 172.21.15.98:0/3389254579 learned_addr learned my addr 172.21.15.98:0/3389254579 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377335730 0 0) 0x7fec24157e20 con 0x7fec240adba0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec241582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec29618700 1 --2- 172.21.15.98:0/3389254579 >> v2:172.21.15.184:3300/0 conn(0x7fec240ce8e0 0x7fec2415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec14003570 con 0x7fec240adba0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2290255634 0 0) 0x7fec14003570 con 0x7fec240adba0 2023-10-06T01:41:45.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 msgr2=0x7fec241582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec241582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 >> v2:172.21.15.184:3300/0 conn(0x7fec240ce8e0 msgr2=0x7fec2415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 --2- 172.21.15.98:0/3389254579 >> v2:172.21.15.184:3300/0 conn(0x7fec240ce8e0 0x7fec2415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240cb8f0 con 0x7fec240adba0 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7fec10001470 con 0x7fec240adba0 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 >> v1:172.21.15.98:6789/0 conn(0x7fec240adba0 legacy=0x7fec240ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec14003a60 0x7fec14003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14004d50 0x7fec140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240cbbd0 con 0x7fec14003a60 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240cb610 con 0x7fec14004d50 2023-10-06T01:41:45.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1400ba60 con 0x7fec240ca3b0 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec14003a60 0x7fec14003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec29618700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14004d50 0x7fec140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec23fff700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 -- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 msgr2=0x7fec14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240ca3b0 0x7fec14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 -- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14004d50 msgr2=0x7fec140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14004d50 0x7fec140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec28e17700 1 -- 172.21.15.98:0/3389254579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14003570 con 0x7fec14003a60 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.224+0000 7fec29618700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14004d50 0x7fec140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec28e17700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec14003a60 0x7fec14003e70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fec18009c00 tx=0x7fec180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435168ba1e20bf6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800e6b0 con 0x7fec14003a60 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800e810 con 0x7fec14003a60 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec237fe700 1 -- 172.21.15.98:0/3389254579 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800ebd0 con 0x7fec14003a60 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 -- 172.21.15.98:0/3389254579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec14003a60 msgr2=0x7fec14003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 --2- 172.21.15.98:0/3389254579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec14003a60 0x7fec14003e70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fec18009c00 tx=0x7fec180091a0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec29618700 1 -- 172.21.15.98:0/3389254579 reap_dead start 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 -- 172.21.15.98:0/3389254579 shutdown_connections 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 -- 172.21.15.98:0/3389254579 >> 172.21.15.98:0/3389254579 conn(0x7fec24070810 msgr2=0x7fec24070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 -- 172.21.15.98:0/3389254579 shutdown_connections 2023-10-06T01:41:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.225+0000 7fec2b07b700 1 -- 172.21.15.98:0/3389254579 wait complete. 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.226+0000 7fec2b07b700 1 Processor -- start 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 -- start start 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14003a60 0x7fec2415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240b1520 0x7fec24148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2415b6e0 con 0x7fec2415b270 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2415c510 con 0x7fec14003a60 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec2b07b700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240cd750 con 0x7fec240b1520 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51174/0 (socket says 172.21.15.98:51174) 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.227+0000 7fec23fff700 1 -- 172.21.15.98:0/212649914 learned_addr learned my addr 172.21.15.98:0/212649914 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec28e17700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14003a60 0x7fec2415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 -- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240b1520 msgr2=0x7fec24148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240b1520 0x7fec24148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 -- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14003a60 msgr2=0x7fec2415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14003a60 0x7fec2415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 -- 172.21.15.98:0/212649914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2415b900 con 0x7fec2415b270 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec23fff700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fec1000e990 tx=0x7fec10001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000c810 con 0x7fec2415b270 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1000c990 con 0x7fec2415b270 2023-10-06T01:41:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2415bbe0 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000cd50 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.228+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2415c1a0 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.229+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec24064ad0 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.229+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec10013630 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.230+0000 7fec20ff9700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 0x7fec08069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.230+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fec100845c0 con 0x7fec2415b270 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.230+0000 7fec28e17700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 0x7fec08069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.230+0000 7fec28e17700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 0x7fec08069410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec18006ed0 tx=0x7fec1801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.236 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.234+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec10051830 con 0x7fec2415b270 2023-10-06T01:41:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.416+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fec2415c650 con 0x7fec08066f60 2023-10-06T01:41:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.416+0000 7fec20ff9700 1 -- 172.21.15.98:0/212649914 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fec2415c650 con 0x7fec08066f60 2023-10-06T01:41:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 msgr2=0x7fec08069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 0x7fec08069410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec18006ed0 tx=0x7fec1801a000 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 msgr2=0x7fec240cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fec1000e990 tx=0x7fec10001df0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 shutdown_connections 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fec08066f60 0x7fec08069410 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec240b1520 0x7fec24148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec14003a60 0x7fec2415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 --2- 172.21.15.98:0/212649914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec2415b270 0x7fec240cd210 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.418+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 >> 172.21.15.98:0/212649914 conn(0x7fec24070810 msgr2=0x7fec24154590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.419+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 shutdown_connections 2023-10-06T01:41:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.419+0000 7fec2b07b700 1 -- 172.21.15.98:0/212649914 wait complete. 2023-10-06T01:41:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T01:41:45.431 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T01:41:45.431 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:45.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.547+0000 7fa8786ed700 1 Processor -- start 2023-10-06T01:41:45.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.547+0000 7fa8786ed700 1 -- start start 2023-10-06T01:41:45.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa87006ed20 0x7fa8700b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 0x7fa8700b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8700aee20 0x7fa8700af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87006fb80 con 0x7fa87006ed20 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700b2d20 con 0x7fa8700aee20 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa8786ed700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700b0450 con 0x7fa87006f0f0 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa876489700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa87006ed20 0x7fa8700b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa876489700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa87006ed20 0x7fa8700b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51190/0 (socket says 172.21.15.98:51190) 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa876489700 1 -- 172.21.15.98:0/2980501428 learned_addr learned my addr 172.21.15.98:0/2980501428 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.548+0000 7fa876c8a700 1 --2- 172.21.15.98:0/2980501428 >> v2:172.21.15.184:3300/0 conn(0x7fa8700aee20 0x7fa8700af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa875c88700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 0x7fa8700b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 -- 172.21.15.98:0/2980501428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 msgr2=0x7fa8700b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 0x7fa8700b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 -- 172.21.15.98:0/2980501428 >> v2:172.21.15.184:3300/0 conn(0x7fa8700aee20 msgr2=0x7fa8700af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 --2- 172.21.15.98:0/2980501428 >> v2:172.21.15.184:3300/0 conn(0x7fa8700aee20 0x7fa8700af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 -- 172.21.15.98:0/2980501428 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700af7c0 con 0x7fa87006ed20 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa875c88700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 0x7fa8700b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa876489700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87006ed20 0x7fa8700b23a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b1390 tx=0x7fa86c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62022bcbf10b7819 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa875487700 1 -- 172.21.15.98:0/2980501428 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00e070 con 0x7fa87006ed20 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa875487700 1 -- 172.21.15.98:0/2980501428 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c00ba50 con 0x7fa87006ed20 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa875487700 1 -- 172.21.15.98:0/2980501428 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0092e0 con 0x7fa87006ed20 2023-10-06T01:41:45.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa8786ed700 1 -- 172.21.15.98:0/2980501428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87006ed20 msgr2=0x7fa8700b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa8786ed700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87006ed20 0x7fa8700b23a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b1390 tx=0x7fa86c009840 comp rx=0 tx=0).stop 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa8786ed700 1 -- 172.21.15.98:0/2980501428 shutdown_connections 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.549+0000 7fa8786ed700 1 --2- 172.21.15.98:0/2980501428 >> v2:172.21.15.184:3300/0 conn(0x7fa8700aee20 0x7fa8700af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.550+0000 7fa8786ed700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87006ed20 0x7fa8700b23a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.550+0000 7fa8786ed700 1 --2- 172.21.15.98:0/2980501428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006f0f0 0x7fa8700b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.550+0000 7fa8786ed700 1 -- 172.21.15.98:0/2980501428 >> 172.21.15.98:0/2980501428 conn(0x7fa870068330 msgr2=0x7fa870068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.550+0000 7fa8786ed700 1 -- 172.21.15.98:0/2980501428 shutdown_connections 2023-10-06T01:41:45.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.550+0000 7fa8786ed700 1 -- 172.21.15.98:0/2980501428 wait complete. 2023-10-06T01:41:45.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 Processor -- start 2023-10-06T01:41:45.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 -- start start 2023-10-06T01:41:45.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006ed20 0x7fa870153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.556 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa87006f0f0 0x7fa870155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 0x7fa87015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870155590 con 0x7fa8700aee20 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870154980 con 0x7fa87006f0f0 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa8786ed700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa870154c60 con 0x7fa87006ed20 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa876c8a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 0x7fa87015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa876c8a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 0x7fa87015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51196/0 (socket says 172.21.15.98:51196) 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.551+0000 7fa876c8a700 1 -- 172.21.15.98:0/1111235555 learned_addr learned my addr 172.21.15.98:0/1111235555 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 -- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006ed20 msgr2=0x7fa870153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa87006ed20 0x7fa870153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.557 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 -- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa87006f0f0 msgr2=0x7fa870155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa875c88700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa87006f0f0 0x7fa870155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa87006f0f0 0x7fa870155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c009ae0 con 0x7fa8700aee20 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa876c8a700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 0x7fa87015f860 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa86800d890 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8680125f0 con 0x7fa8700aee20 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868012770 con 0x7fa8700aee20 2023-10-06T01:41:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa870154f40 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868012b30 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.552+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701545c0 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.553+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa870064ad0 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.553+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa86801e460 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa8667fc700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa854066f30 0x7fa8540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fa868084c20 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa876489700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa854066f30 0x7fa8540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa8667fc700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fa85406d8f0 0x7fa85406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8540703f0 con 0x7fa85406d8f0 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa868087260 con 0x7fa8700aee20 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa876489700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa854066f30 0x7fa8540693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b1390 tx=0x7fa86c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.554+0000 7fa875c88700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fa85406d8f0 0x7fa85406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.555+0000 7fa875c88700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fa85406d8f0 0x7fa85406fd20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.560 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.555+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== osd.6 v2:172.21.15.184:6800/120875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa8540703f0 con 0x7fa85406d8f0 2023-10-06T01:41:45.569 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 --> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa8700b2a40 con 0x7fa85406d8f0 2023-10-06T01:41:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8667fc700 1 -- 172.21.15.98:0/1111235555 <== osd.6 v2:172.21.15.184:6800/120875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa8700b2a40 con 0x7fa85406d8f0 2023-10-06T01:41:45.570 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:45.570 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:45.570 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fa85406d8f0 msgr2=0x7fa85406fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.184:6800/120875,v1:172.21.15.184:6801/120875] conn(0x7fa85406d8f0 0x7fa85406fd20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa854066f30 msgr2=0x7fa8540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fa854066f30 0x7fa8540693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b1390 tx=0x7fa86c018000 comp rx=0 tx=0).stop 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 msgr2=0x7fa87015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.569+0000 7fa8786ed700 1 --2- 172.21.15.98:0/1111235555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8700aee20 0x7fa87015f860 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa86800d890 tx=0x7fa868008040 comp rx=0 tx=0).stop 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.570+0000 7fa876c8a700 1 -- 172.21.15.98:0/1111235555 reap_dead start 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.570+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 shutdown_connections 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.570+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 >> 172.21.15.98:0/1111235555 conn(0x7fa870068330 msgr2=0x7fa8700b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.570+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 shutdown_connections 2023-10-06T01:41:45.572 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.570+0000 7fa8786ed700 1 -- 172.21.15.98:0/1111235555 wait complete. 2023-10-06T01:41:45.582 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T01:41:45.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.695+0000 7f2cb4bd8700 1 Processor -- start 2023-10-06T01:41:45.696 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.695+0000 7f2cb4bd8700 1 -- start start 2023-10-06T01:41:45.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2cb00cbfd0 0x7f2cb0068b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cb0156690 0x7f2cb0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.698 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb006eb90 con 0x7f2cb0156690 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb0157660 con 0x7f2cb00cbfd0 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2cb4bd8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0157380 con 0x7f2cb0153c70 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cb0156690 0x7f2cb0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cb0156690 0x7f2cb0156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51208/0 (socket says 172.21.15.98:51208) 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 -- 172.21.15.98:0/1017173404 learned_addr learned my addr 172.21.15.98:0/1017173404 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caeffd700 1 --2- 172.21.15.98:0/1017173404 >> v2:172.21.15.184:3300/0 conn(0x7f2cb00cbfd0 0x7f2cb0068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caf7fe700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 -- 172.21.15.98:0/1017173404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 msgr2=0x7f2cb0068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 -- 172.21.15.98:0/1017173404 >> v2:172.21.15.184:3300/0 conn(0x7f2cb00cbfd0 msgr2=0x7f2cb0068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:45.699 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.696+0000 7f2caffff700 1 --2- 172.21.15.98:0/1017173404 >> v2:172.21.15.184:3300/0 conn(0x7f2cb00cbfd0 0x7f2cb0068b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2caffff700 1 -- 172.21.15.98:0/1017173404 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00cb4f0 con 0x7f2cb0156690 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2caf7fe700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2caffff700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb0156af0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0069140 tx=0x7f2ca800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a0a2a531aac3ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2cae7fc700 1 -- 172.21.15.98:0/1017173404 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800dba0 con 0x7f2cb0156690 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2cae7fc700 1 -- 172.21.15.98:0/1017173404 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800dd00 con 0x7f2cb0156690 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2cae7fc700 1 -- 172.21.15.98:0/1017173404 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80196b0 con 0x7f2cb0156690 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/1017173404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 msgr2=0x7f2cb0156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.697+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb0156af0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0069140 tx=0x7f2ca800b4f0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/1017173404 shutdown_connections 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb0156af0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/1017173404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.700 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/1017173404 >> v2:172.21.15.184:3300/0 conn(0x7f2cb00cbfd0 0x7f2cb0068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/1017173404 >> 172.21.15.98:0/1017173404 conn(0x7f2cb00c9c50 msgr2=0x7f2cb006db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/1017173404 shutdown_connections 2023-10-06T01:41:45.701 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.698+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/1017173404 wait complete. 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.699+0000 7f2cb4bd8700 1 Processor -- start 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.699+0000 7f2cb4bd8700 1 -- start start 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.699+0000 7f2cb4bd8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb00cbfd0 0x7f2cb0165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00c3290 con 0x7f2cb0156690 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00af670 con 0x7f2cb00cbfd0 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00af910 con 0x7f2cb0153c70 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51220/0 (socket says 172.21.15.98:51220) 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 -- 172.21.15.98:0/3063822988 learned_addr learned my addr 172.21.15.98:0/3063822988 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caf7fe700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb00cbfd0 0x7f2cb0165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caeffd700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 -- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 msgr2=0x7f2cb0156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 -- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb00cbfd0 msgr2=0x7f2cb0165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb00cbfd0 0x7f2cb0165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 -- 172.21.15.98:0/3063822988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca800b790 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2caffff700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0069610 tx=0x7f2ca801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800dba0 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800de70 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8010da0 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb00afbf0 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.700+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb0143620 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.701+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb00c33d0 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.702+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ca8023620 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.702+0000 7f2c977fe700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.702+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f2ca8014070 con 0x7f2cb0156690 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.702+0000 7f2caf7fe700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.702+0000 7f2caf7fe700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 0x7f2c9c0693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0159000 tx=0x7f2ca0009930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:45.706 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.706+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cb00c33d0 con 0x7f2cb0156690 2023-10-06T01:41:45.892 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.892+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cb00c33d0 con 0x7f2c9c066f30 2023-10-06T01:41:45.893 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.892+0000 7f2c977fe700 1 -- 172.21.15.98:0/3063822988 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2cb00c33d0 con 0x7f2c9c066f30 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 msgr2=0x7f2c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 0x7f2c9c0693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0159000 tx=0x7f2ca0009930 comp rx=0 tx=0).stop 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 msgr2=0x7f2cb01430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0069610 tx=0x7f2ca801b000 comp rx=0 tx=0).stop 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 shutdown_connections 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb0156690 0x7f2cb01430e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb00cbfd0 0x7f2cb0165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.895 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 --2- 172.21.15.98:0/3063822988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb0153c70 0x7f2cb0156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:45.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 >> 172.21.15.98:0/3063822988 conn(0x7f2cb00c9c50 msgr2=0x7f2cb0153240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:45.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 shutdown_connections 2023-10-06T01:41:45.896 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:45.894+0000 7f2cb4bd8700 1 -- 172.21.15.98:0/3063822988 wait complete. 2023-10-06T01:41:45.896 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T01:41:45.906 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T01:41:45.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T01:41:46.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.024+0000 7f216d4fb700 1 Processor -- start 2023-10-06T01:41:46.024 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.024+0000 7f216d4fb700 1 -- start start 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.024+0000 7f216d4fb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f21680abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.024+0000 7f216d4fb700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680ac530 con 0x7f216806ad20 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f216d4fb700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680695d0 con 0x7f21680ccdb0 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f216d4fb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21680ad370 con 0x7f21680abc20 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f21677fe700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f216806ad20 0x7f21680af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42542/0 (socket says 172.21.15.98:42542) 2023-10-06T01:41:46.028 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f21677fe700 1 -- 172.21.15.98:0/181155073 learned_addr learned my addr 172.21.15.98:0/181155073 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2166ffd700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f21680abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598599166 0 0) 0x7f21680ac530 con 0x7f216806ad20 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154003570 con 0x7f216806ad20 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3370408927 0 0) 0x7f2154003570 con 0x7f216806ad20 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 msgr2=0x7f21680abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f21680abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 >> v1:172.21.15.184:6789/0 conn(0x7f21680ccdb0 legacy=0x7f21680a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680ac760 con 0x7f216806ad20 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f2158001470 con 0x7f216806ad20 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 >> v1:172.21.15.98:6789/0 conn(0x7f216806ad20 legacy=0x7f21680af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.029 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f2154003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f2154004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2154004d50 0x7f21540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215400bbf0 con 0x7f2154004010 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215400bd30 con 0x7f2154004d50 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215400e100 con 0x7f21680abc20 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f21667fc700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f2154004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f2166ffd700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f2154003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.025+0000 7f21677fe700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2154004d50 0x7f21540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 -- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 msgr2=0x7f2154003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21680abc20 0x7f2154003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 -- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2154004d50 msgr2=0x7f21540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2154004d50 0x7f21540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 -- 172.21.15.98:0/181155073 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154003570 con 0x7f2154004010 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21667fc700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f2154004810 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f21680cd220 tx=0x7f2150007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff0fc178cd5634 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150012420 con 0x7f2154004010 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150012580 con 0x7f2154004010 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f2165ffb700 1 -- 172.21.15.98:0/181155073 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150012920 con 0x7f2154004010 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f216d4fb700 1 -- 172.21.15.98:0/181155073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 msgr2=0x7f2154004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f216d4fb700 1 --2- 172.21.15.98:0/181155073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f2154004810 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f21680cd220 tx=0x7f2150007d30 comp rx=0 tx=0).stop 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f21677fe700 1 -- 172.21.15.98:0/181155073 reap_dead start 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f216d4fb700 1 -- 172.21.15.98:0/181155073 shutdown_connections 2023-10-06T01:41:46.031 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.026+0000 7f216d4fb700 1 -- 172.21.15.98:0/181155073 >> 172.21.15.98:0/181155073 conn(0x7f21680cc110 msgr2=0x7f21680cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.027+0000 7f216d4fb700 1 -- 172.21.15.98:0/181155073 shutdown_connections 2023-10-06T01:41:46.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.027+0000 7f216d4fb700 1 -- 172.21.15.98:0/181155073 wait complete. 2023-10-06T01:41:46.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 Processor -- start 2023-10-06T01:41:46.032 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 -- start start 2023-10-06T01:41:46.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f216814ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21681523a0 0x7f2168099510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.034 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2168099a50 0x7f216806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168099080 con 0x7f2154004010 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168098470 con 0x7f2168099a50 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.028+0000 7f216d4fb700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2168098750 con 0x7f21681523a0 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f216814ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f216814ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51240/0 (socket says 172.21.15.98:51240) 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 -- 172.21.15.98:0/2935827999 learned_addr learned my addr 172.21.15.98:0/2935827999 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f21677fe700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2168099a50 0x7f216806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f21667fc700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21681523a0 0x7f2168099510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 -- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21681523a0 msgr2=0x7f2168099510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21681523a0 0x7f2168099510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.035 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 -- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2168099a50 msgr2=0x7f216806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2168099a50 0x7f216806d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168098a30 con 0x7f2154004010 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f2166ffd700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f216814ea00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2160011f80 tx=0x7f216000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000e040 con 0x7f2154004010 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216000f040 con 0x7f2154004010 2023-10-06T01:41:46.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168098d10 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160015430 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.029+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216806d8b0 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.030+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2168064ad0 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.030+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21600157e0 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f215effd700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2148066f30 0x7f21480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f21600852e0 con 0x7f2154004010 2023-10-06T01:41:46.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f21667fc700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2148066f30 0x7f21480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f215effd700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7f214806d8f0 0x7f214806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21480703f0 con 0x7f214806d8f0 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2160087920 con 0x7f2154004010 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f21677fe700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7f214806d8f0 0x7f214806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.031+0000 7f21667fc700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2148066f30 0x7f21480693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2150001370 tx=0x7f2150015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.032+0000 7f21677fe700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7f214806d8f0 0x7f214806fd20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.038 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.032+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== osd.7 v2:172.21.15.184:6816/120876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f21480703f0 con 0x7f214806d8f0 2023-10-06T01:41:46.046 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.046+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 --> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f216804bb80 con 0x7f214806d8f0 2023-10-06T01:41:46.047 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f215effd700 1 -- 172.21.15.98:0/2935827999 <== osd.7 v2:172.21.15.184:6816/120876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f216804bb80 con 0x7f214806d8f0 2023-10-06T01:41:46.047 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-10-06T01:41:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T01:41:46.047 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-10-06T01:41:46.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7f214806d8f0 msgr2=0x7f214806fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.048 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.184:6816/120876,v1:172.21.15.184:6817/120876] conn(0x7f214806d8f0 0x7f214806fd20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2148066f30 msgr2=0x7f21480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f2148066f30 0x7f21480693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2150001370 tx=0x7f2150015000 comp rx=0 tx=0).stop 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 msgr2=0x7f216814ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 --2- 172.21.15.98:0/2935827999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2154004010 0x7f216814ea00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2160011f80 tx=0x7f216000b040 comp rx=0 tx=0).stop 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f21677fe700 1 -- 172.21.15.98:0/2935827999 reap_dead start 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 shutdown_connections 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 >> 172.21.15.98:0/2935827999 conn(0x7f21680cc110 msgr2=0x7f21680b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 shutdown_connections 2023-10-06T01:41:46.049 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.047+0000 7f216d4fb700 1 -- 172.21.15.98:0/2935827999 wait complete. 2023-10-06T01:41:46.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T01:41:46.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.174+0000 7f14ae8c5700 1 Processor -- start 2023-10-06T01:41:46.175 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 -- start start 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14a80ca3b0 0x7f14a81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a80adba0 0x7f14a80ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a80cbbd0 con 0x7f14a80adba0 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a8157e20 con 0x7f14a80ca3b0 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14ae8c5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80ceb50 con 0x7f14a80ce710 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14a77fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a80adba0 0x7f14a80ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14a77fe700 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a80adba0 0x7f14a80ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51244/0 (socket says 172.21.15.98:51244) 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.175+0000 7f14a77fe700 1 -- 172.21.15.98:0/4058380917 learned_addr learned my addr 172.21.15.98:0/4058380917 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 -- 172.21.15.98:0/4058380917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 msgr2=0x7f14a815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.178 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 --2- 172.21.15.98:0/4058380917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 -- 172.21.15.98:0/4058380917 >> v2:172.21.15.184:3300/0 conn(0x7f14a80ca3b0 msgr2=0x7f14a81582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a7fff700 1 --2- 172.21.15.98:0/4058380917 >> v2:172.21.15.184:3300/0 conn(0x7f14a80ca3b0 0x7f14a81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 --2- 172.21.15.98:0/4058380917 >> v2:172.21.15.184:3300/0 conn(0x7f14a80ca3b0 0x7f14a81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 -- 172.21.15.98:0/4058380917 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a80cb610 con 0x7f14a80adba0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a77fe700 1 --2- 172.21.15.98:0/4058380917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80adba0 0x7f14a80ce1d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f14a806f300 tx=0x7f149c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05b0aa5f0c67603 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a6ffd700 1 -- 172.21.15.98:0/4058380917 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00b860 con 0x7f14a80adba0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a6ffd700 1 -- 172.21.15.98:0/4058380917 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00b9c0 con 0x7f14a80adba0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.176+0000 7f14a6ffd700 1 -- 172.21.15.98:0/4058380917 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00bd60 con 0x7f14a80adba0 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 -- 172.21.15.98:0/4058380917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80adba0 msgr2=0x7f14a80ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/4058380917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80adba0 0x7f14a80ce1d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f14a806f300 tx=0x7f149c009250 comp rx=0 tx=0).stop 2023-10-06T01:41:46.179 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 -- 172.21.15.98:0/4058380917 shutdown_connections 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/4058380917 >> v2:172.21.15.184:3300/0 conn(0x7f14a80ca3b0 0x7f14a81582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/4058380917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/4058380917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80adba0 0x7f14a80ce1d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 -- 172.21.15.98:0/4058380917 >> 172.21.15.98:0/4058380917 conn(0x7f14a8070810 msgr2=0x7f14a8070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 -- 172.21.15.98:0/4058380917 shutdown_connections 2023-10-06T01:41:46.180 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.177+0000 7f14ae8c5700 1 -- 172.21.15.98:0/4058380917 wait complete. 2023-10-06T01:41:46.182 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.178+0000 7f14ae8c5700 1 Processor -- start 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.178+0000 7f14ae8c5700 1 -- start start 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a80adba0 0x7f14a80c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a81674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80b1ae0 con 0x7f14a80ca3b0 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80c4060 con 0x7f14a80adba0 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80b1800 con 0x7f14a80ce710 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51250/0 (socket says 172.21.15.98:51250) 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 -- 172.21.15.98:0/2018087909 learned_addr learned my addr 172.21.15.98:0/2018087909 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:46.183 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 -- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 msgr2=0x7f14a81674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a81674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 -- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a80adba0 msgr2=0x7f14a80c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a80adba0 0x7f14a80c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 -- 172.21.15.98:0/2018087909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0094f0 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14a77fe700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f149c009290 tx=0x7f149c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c026480 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c01f070 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a80c3a10 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c026830 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.179+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80c5860 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.180+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a8064ad0 con 0x7f14a80ca3b0 2023-10-06T01:41:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.181+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f149c026ba0 con 0x7f14a80ca3b0 2023-10-06T01:41:46.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.181+0000 7f148ffff700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.181+0000 7f14a7fff700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.181+0000 7f14a7fff700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14a816aa80 tx=0x7f1498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.185 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.182+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f149c021070 con 0x7f14a80ca3b0 2023-10-06T01:41:46.186 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.186+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f149c053600 con 0x7f14a80ca3b0 2023-10-06T01:41:46.369 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.369+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f14a80c3890 con 0x7f1494066f30 2023-10-06T01:41:46.370 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.369+0000 7f148ffff700 1 -- 172.21.15.98:0/2018087909 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1498007550 con 0x7f1494066f30 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 msgr2=0x7f14940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14a816aa80 tx=0x7f1498008040 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 msgr2=0x7f14a80c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f149c009290 tx=0x7f149c017000 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 shutdown_connections 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a80ce710 0x7f14a81674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a80adba0 0x7f14a80c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 --2- 172.21.15.98:0/2018087909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a80ca3b0 0x7f14a80c5320 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.372 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 >> 172.21.15.98:0/2018087909 conn(0x7f14a8070810 msgr2=0x7f14a815a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 shutdown_connections 2023-10-06T01:41:46.373 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.371+0000 7f14ae8c5700 1 -- 172.21.15.98:0/2018087909 wait complete. 2023-10-06T01:41:46.373 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T01:41:46.383 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:41:46.497 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.497+0000 7f6ba1d75700 1 Processor -- start 2023-10-06T01:41:46.498 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.497+0000 7f6ba1d75700 1 -- start start 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6ba1d75700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6b9c0b07a0 0x7f6b9c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6ba1d75700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9c0b0ca0 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6ba1d75700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9c0b0f80 con 0x7f6b9c1595a0 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6ba1d75700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0ab390 con 0x7f6b9c0b07a0 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9affd700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6b9c0b49e0 0x7f6b9c0aac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42556/0 (socket says 172.21.15.98:42556) 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9affd700 1 -- 172.21.15.98:0/2917607602 learned_addr learned my addr 172.21.15.98:0/2917607602 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:41:46.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9bfff700 1 --2- 172.21.15.98:0/2917607602 >> v2:172.21.15.184:3300/0 conn(0x7f6b9c0b07a0 0x7f6b9c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715396727 0 0) 0x7f6b9c0b0ca0 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b88003570 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662640056 0 0) 0x7f6b9c0b0f80 con 0x7f6b9c1595a0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b9c0b0ca0 con 0x7f6b9c1595a0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807773296 0 0) 0x7f6b88003570 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 >> v2:172.21.15.184:3300/0 conn(0x7f6b9c0b07a0 msgr2=0x7f6b9c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 --2- 172.21.15.98:0/2917607602 >> v2:172.21.15.184:3300/0 conn(0x7f6b9c0b07a0 0x7f6b9c0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 >> v1:172.21.15.184:6789/0 conn(0x7f6b9c1595a0 legacy=0x7f6b9c0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0b1260 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f6b84001470 con 0x7f6b9c0b49e0 2023-10-06T01:41:46.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 >> v1:172.21.15.98:6789/0 conn(0x7f6b9c0b49e0 legacy=0x7f6b9c0aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 0x7f6b88003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b88004810 0x7f6b88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b880051c0 0x7f6b8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0ab390 con 0x7f6b88003bf0 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8800c0b0 con 0x7f6b88004810 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.498+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8800c270 con 0x7f6b880051c0 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 0x7f6b88003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9affd700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b88004810 0x7f6b88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9bfff700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b880051c0 0x7f6b8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 -- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b880051c0 msgr2=0x7f6b8800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b880051c0 0x7f6b8800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 -- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b88004810 msgr2=0x7f6b88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b88004810 0x7f6b88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 -- 172.21.15.98:0/2917607602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88003570 con 0x7f6b88003bf0 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9bfff700 1 -- 172.21.15.98:0/2917607602 reap_dead start 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 0x7f6b88003fe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3c0 tx=0x7f6b94001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2352c8dd92b8100f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94001ee0 con 0x7f6b88003bf0 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9400cc50 con 0x7f6b88003bf0 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.499+0000 7f6b9a7fc700 1 -- 172.21.15.98:0/2917607602 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94012420 con 0x7f6b88003bf0 2023-10-06T01:41:46.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 -- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 msgr2=0x7f6b88003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 0x7f6b88003fe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3c0 tx=0x7f6b94001bd0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 -- 172.21.15.98:0/2917607602 shutdown_connections 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/2917607602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b88003bf0 0x7f6b88003fe0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 -- 172.21.15.98:0/2917607602 >> 172.21.15.98:0/2917607602 conn(0x7f6b9c099830 msgr2=0x7f6b9c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 -- 172.21.15.98:0/2917607602 shutdown_connections 2023-10-06T01:41:46.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.500+0000 7f6ba1d75700 1 -- 172.21.15.98:0/2917607602 wait complete. 2023-10-06T01:41:46.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.501+0000 7f6ba1d75700 1 Processor -- start 2023-10-06T01:41:46.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.501+0000 7f6ba1d75700 1 -- start start 2023-10-06T01:41:46.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b88003bf0 0x7f6b9c09b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.508 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 0x7f6b9c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c09ce60 con 0x7f6b9c09bd10 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c09c250 con 0x7f6b9c15ae80 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c09c530 con 0x7f6b88003bf0 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51278/0 (socket says 172.21.15.98:51278) 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 -- 172.21.15.98:0/221288950 learned_addr learned my addr 172.21.15.98:0/221288950 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b88003bf0 0x7f6b9c09b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9bfff700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 0x7f6b9c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 -- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b88003bf0 msgr2=0x7f6b9c09b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b88003bf0 0x7f6b9c09b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 -- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 msgr2=0x7f6b9c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 0x7f6b9c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 -- 172.21.15.98:0/221288950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c09c810 con 0x7f6b9c09bd10 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9bfff700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 0x7f6b9c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b9affd700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b840011e0 tx=0x7f6b84001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84018030 con 0x7f6b9c09bd10 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b84000c70 con 0x7f6b9c09bd10 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c09caf0 con 0x7f6b9c09bd10 2023-10-06T01:41:46.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400fb40 con 0x7f6b9c09bd10 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.502+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c15bb70 con 0x7f6b9c09bd10 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.503+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9c064ad0 con 0x7f6b9c09bd10 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.504+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b8401b070 con 0x7f6b9c09bd10 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.504+0000 7f6b937fe700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 0x7f6b7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.504+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f6b84019070 con 0x7f6b9c09bd10 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.504+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 0x7f6b7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.504+0000 7f6b9b7fe700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 0x7f6b7c0693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3c0 tx=0x7f6b94016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:41:46.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.508+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b84051640 con 0x7f6b9c09bd10 2023-10-06T01:41:46.689 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.688+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b9c0b1540 con 0x7f6b7c066f30 2023-10-06T01:41:46.691 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.691+0000 7f6b937fe700 1 -- 172.21.15.98:0/221288950 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73249 (secure 0 0 0) 0x7f6b9c0b1540 con 0x7f6b7c066f30 2023-10-06T01:41:46.692 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 msgr2=0x7f6b7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 0x7f6b7c0693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3c0 tx=0x7f6b94016000 comp rx=0 tx=0).stop 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 msgr2=0x7f6b9c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b840011e0 tx=0x7f6b84001df0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 shutdown_connections 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6b7c066f30 0x7f6b7c0693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c15ae80 0x7f6b9c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b88003bf0 0x7f6b9c09b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.694 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 --2- 172.21.15.98:0/221288950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b9c09bd10 0x7f6b9c15a940 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:41:46.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 >> 172.21.15.98:0/221288950 conn(0x7f6b9c099830 msgr2=0x7f6b9c069060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:41:46.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 shutdown_connections 2023-10-06T01:41:46.695 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:41:46.693+0000 7f6ba1d75700 1 -- 172.21.15.98:0/221288950 wait complete. 2023-10-06T01:41:46.695 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:41:46.705 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2023-10-06T01:41:45.810462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384132,"kb_used_data":5828,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590588,"statfs":{"total":773094113280,"available":770652762112,"internally_reserved":0,"allocated":5967872,"data_stored":3417983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6291456,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-16,"num_read_kb":-9,"num_write":-8,"num_write_kb":-6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.656542"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.158198+0000","last_change":"2023-10-06T01:39:39.754200+0000","last_active":"2023-10-06T01:41:36.158198+0000","last_peered":"2023-10-06T01:41:36.158198+0000","last_clean":"2023-10-06T01:41:36.158198+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:36.158198+0000","last_undegraded":"2023-10-06T01:41:36.158198+0000","last_fullsized":"2023-10-06T01:41:36.158198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.274908+0000","last_change":"2023-10-06T01:39:39.296525+0000","last_active":"2023-10-06T01:41:36.274908+0000","last_peered":"2023-10-06T01:41:36.274908+0000","last_clean":"2023-10-06T01:41:36.274908+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:36.274908+0000","last_undegraded":"2023-10-06T01:41:36.274908+0000","last_fullsized":"2023-10-06T01:41:36.274908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275442+0000","last_change":"2023-10-06T01:39:39.293231+0000","last_active":"2023-10-06T01:41:36.275442+0000","last_peered":"2023-10-06T01:41:36.275442+0000","last_clean":"2023-10-06T01:41:36.275442+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:36.275442+0000","last_undegraded":"2023-10-06T01:41:36.275442+0000","last_fullsized":"2023-10-06T01:41:36.275442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:32.937769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":187,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275257+0000","last_change":"2023-10-06T01:39:39.302487+0000","last_active":"2023-10-06T01:41:36.275257+0000","last_peered":"2023-10-06T01:41:36.275257+0000","last_clean":"2023-10-06T01:41:36.275257+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:36.275257+0000","last_undegraded":"2023-10-06T01:41:36.275257+0000","last_fullsized":"2023-10-06T01:41:36.275257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:01.175082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.276533+0000","last_change":"2023-10-06T01:39:39.756515+0000","last_active":"2023-10-06T01:41:36.276533+0000","last_peered":"2023-10-06T01:41:36.276533+0000","last_clean":"2023-10-06T01:41:36.276533+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:36.276533+0000","last_undegraded":"2023-10-06T01:41:36.276533+0000","last_fullsized":"2023-10-06T01:41:36.276533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:19.508141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275156+0000","last_change":"2023-10-06T01:39:39.754016+0000","last_active":"2023-10-06T01:41:36.275156+0000","last_peered":"2023-10-06T01:41:36.275156+0000","last_clean":"2023-10-06T01:41:36.275156+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:36.275156+0000","last_undegraded":"2023-10-06T01:41:36.275156+0000","last_fullsized":"2023-10-06T01:41:36.275156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.275564+0000","last_change":"2023-10-06T01:39:39.754018+0000","last_active":"2023-10-06T01:41:36.275564+0000","last_peered":"2023-10-06T01:41:36.275564+0000","last_clean":"2023-10-06T01:41:36.275564+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:36.275564+0000","last_undegraded":"2023-10-06T01:41:36.275564+0000","last_fullsized":"2023-10-06T01:41:36.275564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:17.560504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.158377+0000","last_change":"2023-10-06T01:39:39.756362+0000","last_active":"2023-10-06T01:41:36.158377+0000","last_peered":"2023-10-06T01:41:36.158377+0000","last_clean":"2023-10-06T01:41:36.158377+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:36.158377+0000","last_undegraded":"2023-10-06T01:41:36.158377+0000","last_fullsized":"2023-10-06T01:41:36.158377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:06.762932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-06T01:41:36.159724+0000","last_change":"2023-10-06T01:39:39.293003+0000","last_active":"2023-10-06T01:41:36.159724+0000","last_peered":"2023-10-06T01:41:36.159724+0000","last_clean":"2023-10-06T01:41:36.159724+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:36.159724+0000","last_undegraded":"2023-10-06T01:41:36.159724+0000","last_fullsized":"2023-10-06T01:41:36.159724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:39:38.268326+0000","last_clean_scrub_stamp":"2023-10-06T01:39:38.268326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:03:46.509745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298452,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073388,"statfs":{"total":96636764160,"available":96331149312,"internally_reserved":0,"allocated":1191936,"data_stored":870193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 01:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45800000000000002,"15min":0.45200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.505},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91500000000000004,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.74299999999999999},"min":{"1min":0.40899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78900000000000003,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.74299999999999999,"15min":0.78200000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63300000000000001,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.73799999999999999,"15min":0.76800000000000002},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.98699999999999999,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.997,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69299999999999995,"15min":0.71499999999999997},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80600000000000005,"15min":0.84399999999999997},"min":{"1min":0.46500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.012,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69299999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48499999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298452,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073388,"statfs":{"total":96636764160,"available":96331149312,"internally_reserved":0,"allocated":1191936,"data_stored":870193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.26400000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.315},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.246}]},{"osd":3,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39800000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.5},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.318,"15min":0.318},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"last":0.23899999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.26200000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.318},{"interface":"front","average":{"1min":0.38,"5min":0.372,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.28499999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.376},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.502}]},{"osd":5,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.439},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.371}]},{"osd":6,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.42799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":278472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.442,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.375,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.47299999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.61399999999999999,"15min":0.65800000000000003},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46300000000000002,"5min":7.383,"15min":7.383},"last":0.314},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.51800000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.55300000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.67500000000000004,"15min":0.72199999999999998},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.51600000000000001,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.82699999999999996,"15min":0.873},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76800000000000002,"5min":7.444,"15min":7.444},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83999999999999997,"5min":1.081,"15min":1.081},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.38900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.86099999999999999,"15min":0.90400000000000003},"min":{"1min":0.42199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78700000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.623},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85899999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24099999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.52600000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.30499999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39000000000000001,"15min":0.39100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.70599999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73799999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.437,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.498}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.9790000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.90500000000000003,"15min":0.93799999999999994},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.2890000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":0.68500000000000005,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":15.285,"5min":15.285,"15min":15.285},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.7250000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.81599999999999995,"15min":0.84099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3650000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.67400000000000004,"15min":0.60399999999999998},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":11.441000000000001,"5min":11.441000000000001,"15min":11.441000000000001},"last":0.432}]},{"osd":3,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.3330000000000002,"5min":7.452,"15min":7.452},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.012,"15min":0.94199999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":17.087,"5min":17.087,"15min":17.087},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.60599999999999998,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.1470000000000002,"5min":7.1470000000000002,"15min":7.1470000000000002},"last":0.621}]},{"osd":6,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":0.67700000000000005,"15min":0.57499999999999996},"min":{"1min":0.34999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":15.339,"5min":15.339,"15min":15.339},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.64100000000000001,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":7.0999999999999996,"5min":7.0999999999999996,"15min":7.0999999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.321,"5min":0.74099999999999999,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":15.163,"5min":15.163,"15min":15.163},"last":0.61199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.752,"15min":0.752},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61199999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.40999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.46800000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71399999999999997,"5min":0.75,"15min":0.75},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49199999999999999,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:41:46.710 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.710 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.710 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T01:39:38.268326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=39, tm_sec=38, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=1, tm_min=41, tm_sec=41, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T01:41:46.711 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T01:41:50.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:41:50.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:42:06.713 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T01:42:06.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.837+0000 7f653841e700 1 Processor -- start 2023-10-06T01:42:06.838 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.837+0000 7f653841e700 1 -- start start 2023-10-06T01:42:06.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.838+0000 7f653841e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f65300b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.846 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.838+0000 7f653841e700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65300cbfe0 con 0x7f6530159870 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.838+0000 7f653841e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65300cb3d0 con 0x7f65300b29f0 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.838+0000 7f653841e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65300cb6b0 con 0x7f65300b2620 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65369bb700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6530159870 0x7f65300cc470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53230/0 (socket says 172.21.15.98:53230) 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65369bb700 1 -- 172.21.15.98:0/812264821 learned_addr learned my addr 172.21.15.98:0/812264821 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65361ba700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f65300b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960740881 0 0) 0x7f65300cbfe0 con 0x7f6530159870 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f651c003570 con 0x7f6530159870 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796441803 0 0) 0x7f65300cb3d0 con 0x7f65300b29f0 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65300cbfe0 con 0x7f65300b29f0 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276319557 0 0) 0x7f651c003570 con 0x7f6530159870 2023-10-06T01:42:06.847 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 msgr2=0x7f65300b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f65300b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 >> v1:172.21.15.184:6789/0 conn(0x7f65300b29f0 legacy=0x7f6530159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300cb990 con 0x7f6530159870 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f65280017f0 con 0x7f6530159870 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 >> v1:172.21.15.98:6789/0 conn(0x7f6530159870 legacy=0x7f65300cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f651c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c004810 0x7f651c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f651c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300cb6b0 con 0x7f651c003c20 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c00bb40 con 0x7f651c004810 2023-10-06T01:42:06.848 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c00bd00 con 0x7f65300b2620 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65361ba700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f651c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65359b9700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c004810 0x7f651c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.839+0000 7f65369bb700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f651c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 -- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 msgr2=0x7f651c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65300b2620 0x7f651c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 -- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c004810 msgr2=0x7f651c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c004810 0x7f651c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 -- 172.21.15.98:0/812264821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c003570 con 0x7f651c003c20 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65361ba700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f651c004010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f652c009bf0 tx=0x7f652c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc4c34a8ae3916 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:06.849 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c005a00 con 0x7f651c003c20 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c005b60 con 0x7f651c003c20 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65351b8700 1 -- 172.21.15.98:0/812264821 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00e710 con 0x7f651c003c20 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f653841e700 1 -- 172.21.15.98:0/812264821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 msgr2=0x7f651c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f653841e700 1 --2- 172.21.15.98:0/812264821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f651c004010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f652c009bf0 tx=0x7f652c009120 comp rx=0 tx=0).stop 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.840+0000 7f65369bb700 1 -- 172.21.15.98:0/812264821 reap_dead start 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.841+0000 7f653841e700 1 -- 172.21.15.98:0/812264821 shutdown_connections 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.841+0000 7f653841e700 1 -- 172.21.15.98:0/812264821 >> 172.21.15.98:0/812264821 conn(0x7f653006ed20 msgr2=0x7f653006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.841+0000 7f653841e700 1 -- 172.21.15.98:0/812264821 shutdown_connections 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.841+0000 7f653841e700 1 -- 172.21.15.98:0/812264821 wait complete. 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 Processor -- start 2023-10-06T01:42:06.850 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 -- start start 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65300c7510 0x7f65300c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f653015d230 0x7f653015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653015dc10 con 0x7f651c003c20 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300c7f60 con 0x7f65300c7510 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f653841e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65300c8240 con 0x7f653015d230 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.842+0000 7f65361ba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41376/0 (socket says 172.21.15.98:41376) 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 -- 172.21.15.98:0/2675257958 learned_addr learned my addr 172.21.15.98:0/2675257958 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65369bb700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f653015d230 0x7f653015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.851 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65359b9700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65300c7510 0x7f65300c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 -- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f653015d230 msgr2=0x7f653015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f653015d230 0x7f653015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 -- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65300c7510 msgr2=0x7f65300c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65300c7510 0x7f65300c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 -- 172.21.15.98:0/2675257958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300c8520 con 0x7f651c003c20 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65361ba700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f652c00e4b0 tx=0x7f652c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c005a00 con 0x7f651c003c20 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c006240 con 0x7f651c003c20 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65300c8800 con 0x7f651c003c20 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c006600 con 0x7f651c003c20 2023-10-06T01:42:06.852 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.843+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6530152200 con 0x7f651c003c20 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.844+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653015dc10 con 0x7f651c003c20 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.845+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f652c013470 con 0x7f651c003c20 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.845+0000 7f65267fc700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.845+0000 7f65359b9700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.845+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f652c085b60 con 0x7f651c003c20 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.845+0000 7f65359b9700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 0x7f65140693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f653015de50 tx=0x7f6520007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:06.853 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:06.850+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f652c052e90 con 0x7f651c003c20 2023-10-06T01:42:07.030 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.030+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 --> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6530064ad0 con 0x7f6514066f30 2023-10-06T01:42:07.033 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.033+0000 7f65267fc700 1 -- 172.21.15.98:0/2675257958 <== mgr.4104 v2:172.21.15.98:6832/120423 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73196 (secure 0 0 0) 0x7f6530064ad0 con 0x7f6514066f30 2023-10-06T01:42:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 msgr2=0x7f65140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 0x7f65140693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f653015de50 tx=0x7f6520007d30 comp rx=0 tx=0).stop 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 msgr2=0x7f6530151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f652c00e4b0 tx=0x7f652c019000 comp rx=0 tx=0).stop 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 shutdown_connections 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f653015d230 0x7f653015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f651c003c20 0x7f6530151cc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.036 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 --2- 172.21.15.98:0/2675257958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65300c7510 0x7f65300c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 >> 172.21.15.98:0/2675257958 conn(0x7f653006ed20 msgr2=0x7f65300b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:07.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 shutdown_connections 2023-10-06T01:42:07.037 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.035+0000 7f653841e700 1 -- 172.21.15.98:0/2675257958 wait complete. 2023-10-06T01:42:07.037 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2023-10-06T01:42:07.047 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2023-10-06T01:42:05.814740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384132,"kb_used_data":5828,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590588,"statfs":{"total":773094113280,"available":770652762112,"internally_reserved":0,"allocated":5967872,"data_stored":3417983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002667"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:46.874598+0000","last_change":"2023-10-06T01:41:46.874565+0000","last_active":"2023-10-06T01:41:46.874598+0000","last_peered":"2023-10-06T01:41:46.874598+0000","last_clean":"2023-10-06T01:41:46.874598+0000","last_became_active":"2023-10-06T01:39:39.753785+0000","last_became_peered":"2023-10-06T01:39:39.753785+0000","last_unstale":"2023-10-06T01:41:46.874598+0000","last_undegraded":"2023-10-06T01:41:46.874598+0000","last_fullsized":"2023-10-06T01:41:46.874598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:46.874420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:46.874420+0000","last_clean_scrub_stamp":"2023-10-06T01:41:46.874420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:45.103488+0000","scrub_duration":0.001735606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:48.472766+0000","last_change":"2023-10-06T01:41:48.472742+0000","last_active":"2023-10-06T01:41:48.472766+0000","last_peered":"2023-10-06T01:41:48.472766+0000","last_clean":"2023-10-06T01:41:48.472766+0000","last_became_active":"2023-10-06T01:39:39.296304+0000","last_became_peered":"2023-10-06T01:39:39.296304+0000","last_unstale":"2023-10-06T01:41:48.472766+0000","last_undegraded":"2023-10-06T01:41:48.472766+0000","last_fullsized":"2023-10-06T01:41:48.472766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:48.472617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:48.472617+0000","last_clean_scrub_stamp":"2023-10-06T01:41:48.472617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:46.701686+0000","scrub_duration":0.0019855250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:44.702224+0000","last_change":"2023-10-06T01:41:44.702190+0000","last_active":"2023-10-06T01:41:44.702224+0000","last_peered":"2023-10-06T01:41:44.702224+0000","last_clean":"2023-10-06T01:41:44.702224+0000","last_became_active":"2023-10-06T01:39:39.292948+0000","last_became_peered":"2023-10-06T01:39:39.292948+0000","last_unstale":"2023-10-06T01:41:44.702224+0000","last_undegraded":"2023-10-06T01:41:44.702224+0000","last_fullsized":"2023-10-06T01:41:44.702224+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:44.702054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:44.702054+0000","last_clean_scrub_stamp":"2023-10-06T01:41:44.702054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:30:23.411511+0000","scrub_duration":0.001945998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'119","reported_seq":200,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:53.192229+0000","last_change":"2023-10-06T01:41:53.192186+0000","last_active":"2023-10-06T01:41:53.192229+0000","last_peered":"2023-10-06T01:41:53.192229+0000","last_clean":"2023-10-06T01:41:53.192229+0000","last_became_active":"2023-10-06T01:39:39.301688+0000","last_became_peered":"2023-10-06T01:39:39.301688+0000","last_unstale":"2023-10-06T01:41:53.192229+0000","last_undegraded":"2023-10-06T01:41:53.192229+0000","last_fullsized":"2023-10-06T01:41:53.192229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'119","last_scrub_stamp":"2023-10-06T01:41:53.192052+0000","last_deep_scrub":"50'119","last_deep_scrub_stamp":"2023-10-06T01:41:53.192052+0000","last_clean_scrub_stamp":"2023-10-06T01:41:53.192052+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:22.073440+0000","scrub_duration":0.0052884919999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":48,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:52.167400+0000","last_change":"2023-10-06T01:41:52.167367+0000","last_active":"2023-10-06T01:41:52.167400+0000","last_peered":"2023-10-06T01:41:52.167400+0000","last_clean":"2023-10-06T01:41:52.167400+0000","last_became_active":"2023-10-06T01:39:39.756089+0000","last_became_peered":"2023-10-06T01:39:39.756089+0000","last_unstale":"2023-10-06T01:41:52.167400+0000","last_undegraded":"2023-10-06T01:41:52.167400+0000","last_fullsized":"2023-10-06T01:41:52.167400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:52.167272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:52.167272+0000","last_clean_scrub_stamp":"2023-10-06T01:41:52.167272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:36.414958+0000","scrub_duration":0.0025526379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:47.042886+0000","last_change":"2023-10-06T01:41:47.042858+0000","last_active":"2023-10-06T01:41:47.042886+0000","last_peered":"2023-10-06T01:41:47.042886+0000","last_clean":"2023-10-06T01:41:47.042886+0000","last_became_active":"2023-10-06T01:39:39.753707+0000","last_became_peered":"2023-10-06T01:39:39.753707+0000","last_unstale":"2023-10-06T01:41:47.042886+0000","last_undegraded":"2023-10-06T01:41:47.042886+0000","last_fullsized":"2023-10-06T01:41:47.042886+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:47.042769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:47.042769+0000","last_clean_scrub_stamp":"2023-10-06T01:41:47.042769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:16:47.168000+0000","scrub_duration":0.001843812,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:46.268050+0000","last_change":"2023-10-06T01:41:46.268020+0000","last_active":"2023-10-06T01:41:46.268050+0000","last_peered":"2023-10-06T01:41:46.268050+0000","last_clean":"2023-10-06T01:41:46.268050+0000","last_became_active":"2023-10-06T01:39:39.753710+0000","last_became_peered":"2023-10-06T01:39:39.753710+0000","last_unstale":"2023-10-06T01:41:46.268050+0000","last_undegraded":"2023-10-06T01:41:46.268050+0000","last_fullsized":"2023-10-06T01:41:46.268050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:46.267925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:46.267925+0000","last_clean_scrub_stamp":"2023-10-06T01:41:46.267925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:14.735021+0000","scrub_duration":0.0017561339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:48.755824+0000","last_change":"2023-10-06T01:41:48.755779+0000","last_active":"2023-10-06T01:41:48.755824+0000","last_peered":"2023-10-06T01:41:48.755824+0000","last_clean":"2023-10-06T01:41:48.755824+0000","last_became_active":"2023-10-06T01:39:39.756092+0000","last_became_peered":"2023-10-06T01:39:39.756092+0000","last_unstale":"2023-10-06T01:41:48.755824+0000","last_undegraded":"2023-10-06T01:41:48.755824+0000","last_fullsized":"2023-10-06T01:41:48.755824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:48.755681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:48.755681+0000","last_clean_scrub_stamp":"2023-10-06T01:41:48.755681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:31.066143+0000","scrub_duration":0.0022643210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T01:41:46.790102+0000","last_change":"2023-10-06T01:41:46.790074+0000","last_active":"2023-10-06T01:41:46.790102+0000","last_peered":"2023-10-06T01:41:46.790102+0000","last_clean":"2023-10-06T01:41:46.790102+0000","last_became_active":"2023-10-06T01:39:39.292714+0000","last_became_peered":"2023-10-06T01:39:39.292714+0000","last_unstale":"2023-10-06T01:41:46.790102+0000","last_undegraded":"2023-10-06T01:41:46.790102+0000","last_fullsized":"2023-10-06T01:41:46.790102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T01:41:46.789980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T01:41:46.789980+0000","last_clean_scrub_stamp":"2023-10-06T01:41:46.789980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:00.309013+0000","scrub_duration":0.0019949910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298452,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073388,"statfs":{"total":96636764160,"available":96331149312,"internally_reserved":0,"allocated":1191936,"data_stored":870193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.8889999999999998,"5min":3.8889999999999998,"15min":3.8889999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65500000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.75800000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.46300000000000002,"15min":0.45400000000000001},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.31},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39500000000000002}]},{"osd":5,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.437}]},{"osd":6,"last update":"Fri Oct 6 01:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.379},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.46600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45800000000000002,"15min":0.45200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91500000000000004,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.74299999999999999},"min":{"1min":0.40899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78900000000000003,"5min":6.6619999999999999,"15min":6.6619999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.74299999999999999,"15min":0.78200000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63300000000000001,"5min":6.9059999999999997,"15min":6.9059999999999997},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.73799999999999999,"15min":0.76800000000000002},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.98699999999999999,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.997,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69299999999999995,"15min":0.71499999999999997},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80600000000000005,"15min":0.84399999999999997},"min":{"1min":0.46500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.012,"5min":7.6500000000000004,"15min":7.6500000000000004},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69299999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.378},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.34100000000000003}]},{"osd":5,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.499}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75900000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298452,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073388,"statfs":{"total":96636764160,"available":96331149312,"internally_reserved":0,"allocated":1191936,"data_stored":870193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.41299999999999998,"15min":0.35999999999999999},"min":{"1min":0.223,"5min":0.221,"15min":0.221},"max":{"1min":10.122,"5min":10.122,"15min":10.122},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.42799999999999999,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.454000000000001,"5min":10.454000000000001,"15min":10.454000000000001},"last":0.31}]},{"osd":2,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.33300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38100000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.39600000000000002,"15min":0.38500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.433},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.41699999999999998,"15min":0.40000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":4.9930000000000003,"5min":4.9930000000000003,"15min":4.9930000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.434,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.6619999999999999,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 01:41:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8399999999999999,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30499999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.318,"15min":0.318},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.26200000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.379},{"interface":"front","average":{"1min":0.38,"5min":0.372,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.376},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.627},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":278472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.442,"5min":7.3449999999999998,"15min":7.3449999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.375,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.47299999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.61399999999999999,"15min":0.65800000000000003},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46300000000000002,"5min":7.383,"15min":7.383},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.51800000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.55300000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.67500000000000004,"15min":0.72199999999999998},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.51600000000000001,"5min":7.4130000000000003,"15min":7.4130000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.82699999999999996,"15min":0.873},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76800000000000002,"5min":7.444,"15min":7.444},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83999999999999997,"5min":1.081,"15min":1.081},"last":0.622}]},{"osd":5,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.38900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.86099999999999999,"15min":0.90400000000000003},"min":{"1min":0.42199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":7.4710000000000001,"15min":7.4710000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78700000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.85899999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24099999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.52600000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.437}]},{"osd":1,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39000000000000001,"15min":0.39100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.70599999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.62}]},{"osd":6,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73799999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Fri Oct 6 01:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.437,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.9790000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.90500000000000003,"15min":0.93799999999999994},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.2890000000000001,"5min":9.3460000000000001,"15min":9.3460000000000001},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":0.68500000000000005,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":15.285,"5min":15.285,"15min":15.285},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.7250000000000001,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.81599999999999995,"15min":0.84099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3650000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.67400000000000004,"15min":0.60399999999999998},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":11.441000000000001,"5min":11.441000000000001,"15min":11.441000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.3330000000000002,"5min":7.452,"15min":7.452},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.012,"15min":0.94199999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":17.087,"5min":17.087,"15min":17.087},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.60599999999999998,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.1470000000000002,"5min":7.1470000000000002,"15min":7.1470000000000002},"last":0.373}]},{"osd":6,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":0.67700000000000005,"15min":0.57499999999999996},"min":{"1min":0.34999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":15.339,"5min":15.339,"15min":15.339},"last":0.443},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Oct 6 01:41:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.64100000000000001,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":7.0999999999999996,"5min":7.0999999999999996,"15min":7.0999999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.321,"5min":0.74099999999999999,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":15.163,"5min":15.163,"15min":15.163},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.752,"15min":0.752},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61199999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68400000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.40999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.46800000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71399999999999997,"5min":0.75,"15min":0.75},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49199999999999999,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Oct 6 01:41:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T01:42:07.051 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T01:42:07.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 Processor -- start 2023-10-06T01:42:07.163 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 -- start start 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd6280b0680 0x7fd6280ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 0x7fd6280cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6280c5400 con 0x7fd6280cc460 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6280ca7d0 con 0x7fd6280b0680 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.163+0000 7fd62e182700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280ccd40 con 0x7fd6280b0a50 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd6280b0680 0x7fd6280ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd626ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 0x7fd6280cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.165 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd627fff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd6280cc460 0x7fd6280c3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.98:60520/0 (socket says 172.21.15.98:60520) 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd627fff700 1 -- 172.21.15.98:0/1797680036 learned_addr learned my addr 172.21.15.98:0/1797680036 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 -- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 msgr2=0x7fd6280cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 0x7fd6280cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 -- 172.21.15.98:0/1797680036 >> v1:172.21.15.184:6789/0 conn(0x7fd6280cc460 legacy=0x7fd6280c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 -- 172.21.15.98:0/1797680036 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280cd020 con 0x7fd6280b0680 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd626ffd700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 0x7fd6280cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.164+0000 7fd6277fe700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6280ced10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd618009eb0 tx=0x7fd618009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311df2270592a9f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd6267fc700 1 -- 172.21.15.98:0/1797680036 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800e070 con 0x7fd6280b0680 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd6267fc700 1 -- 172.21.15.98:0/1797680036 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61800bc30 con 0x7fd6280b0680 2023-10-06T01:42:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd6267fc700 1 -- 172.21.15.98:0/1797680036 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618012430 con 0x7fd6280b0680 2023-10-06T01:42:07.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 -- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 msgr2=0x7fd6280ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6280ced10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd618009eb0 tx=0x7fd618009780 comp rx=0 tx=0).stop 2023-10-06T01:42:07.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 -- 172.21.15.98:0/1797680036 shutdown_connections 2023-10-06T01:42:07.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6280ced10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.169 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 --2- 172.21.15.98:0/1797680036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280b0a50 0x7fd6280cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 -- 172.21.15.98:0/1797680036 >> 172.21.15.98:0/1797680036 conn(0x7fd628088000 msgr2=0x7fd6280ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 -- 172.21.15.98:0/1797680036 shutdown_connections 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.165+0000 7fd62e182700 1 -- 172.21.15.98:0/1797680036 wait complete. 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.166+0000 7fd62e182700 1 Processor -- start 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.166+0000 7fd62e182700 1 -- start start 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6281554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280cc460 0x7fd628157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628098a40 con 0x7fd6280b0a50 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62809b150 con 0x7fd6280b0680 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd62e182700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6281588c0 con 0x7fd6280cc460 2023-10-06T01:42:07.170 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd626ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd626ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41382/0 (socket says 172.21.15.98:41382) 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd626ffd700 1 -- 172.21.15.98:0/106845063 learned_addr learned my addr 172.21.15.98:0/106845063 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd6277fe700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6281554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd627fff700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280cc460 0x7fd628157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.167+0000 7fd626ffd700 1 -- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280cc460 msgr2=0x7fd628157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd626ffd700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280cc460 0x7fd628157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd626ffd700 1 -- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 msgr2=0x7fd6281554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd626ffd700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6281554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd626ffd700 1 -- 172.21.15.98:0/106845063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c009420 con 0x7fd6280b0a50 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd6277fe700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6281554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd626ffd700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ec80 tx=0x7fd61c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00c850 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c00c9b0 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00cd30 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd618009a20 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.168+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62809ab00 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.169+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62809a960 con 0x7fd6280b0a50 2023-10-06T01:42:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.169+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd61c013680 con 0x7fd6280b0a50 2023-10-06T01:42:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.170+0000 7fd60f7fe700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 0x7fd6140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.170+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd61c084720 con 0x7fd6280b0a50 2023-10-06T01:42:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.170+0000 7fd6277fe700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 0x7fd6140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.170+0000 7fd6277fe700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 0x7fd6140693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd618009eb0 tx=0x7fd618019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.174+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd62809a960 con 0x7fd6280b0a50 2023-10-06T01:42:07.357 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.357+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd62809a960 con 0x7fd6280b0a50 2023-10-06T01:42:07.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.363+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd61c055420 con 0x7fd6280b0a50 2023-10-06T01:42:07.364 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.364+0000 7fd60f7fe700 1 -- 172.21.15.98:0/106845063 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd62809a960 con 0x7fd6280b0a50 2023-10-06T01:42:07.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 msgr2=0x7fd6140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 0x7fd6140693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd618009eb0 tx=0x7fd618019000 comp rx=0 tx=0).stop 2023-10-06T01:42:07.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 msgr2=0x7fd628159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.367 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ec80 tx=0x7fd61c0091a0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 shutdown_connections 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7fd614066f30 0x7fd6140693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6280cc460 0x7fd628157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6280b0680 0x7fd6281554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 --2- 172.21.15.98:0/106845063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6280b0a50 0x7fd628159140 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 >> 172.21.15.98:0/106845063 conn(0x7fd628088000 msgr2=0x7fd6280c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 shutdown_connections 2023-10-06T01:42:07.368 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.366+0000 7fd62e182700 1 -- 172.21.15.98:0/106845063 wait complete. 2023-10-06T01:42:07.379 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T01:42:07.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T01:42:07.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.492+0000 7f14683b1700 1 Processor -- start 2023-10-06T01:42:07.493 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.492+0000 7f14683b1700 1 -- start start 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f14683b1700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14600b49e0 0x7f14600aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f14683b1700 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600b0ca0 con 0x7f14600b07a0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f14683b1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600b0f80 con 0x7f14601595a0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f14683b1700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14600b1260 con 0x7f14600b49e0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f146694e700 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14600b07a0 0x7f14600b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53240/0 (socket says 172.21.15.98:53240) 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f146694e700 1 -- 172.21.15.98:0/2406433088 learned_addr learned my addr 172.21.15.98:0/2406433088 (peer_addr_for_me v1:172.21.15.98:0/0) 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f146594c700 1 --2- 172.21.15.98:0/2406433088 >> v2:172.21.15.184:3300/0 conn(0x7f14600b49e0 0x7f14600aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912659618 0 0) 0x7f14600b0ca0 con 0x7f14600b07a0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.493+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003570 con 0x7f14600b07a0 2023-10-06T01:42:07.500 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327525510 0 0) 0x7f144c003570 con 0x7f14600b07a0 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 >> v2:172.21.15.184:3300/0 conn(0x7f14600b49e0 msgr2=0x7f14600aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 --2- 172.21.15.98:0/2406433088 >> v2:172.21.15.184:3300/0 conn(0x7f14600b49e0 0x7f14600aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 >> v1:172.21.15.184:6789/0 conn(0x7f14601595a0 legacy=0x7f14600aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600b1540 con 0x7f14600b07a0 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2062069462 0 0) 0x7f14580017f0 con 0x7f14600b07a0 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 >> v1:172.21.15.98:6789/0 conn(0x7f14600b07a0 legacy=0x7f14600b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c003a90 0x7f144c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c004810 0x7f144c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.501 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f144c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14600b1260 con 0x7f144c005210 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c130 con 0x7f144c004810 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c2c0 con 0x7f144c003a90 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f144c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146594c700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c004810 0x7f144c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146614d700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c003a90 0x7f144c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 -- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c003a90 msgr2=0x7f144c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c003a90 0x7f144c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 -- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c004810 msgr2=0x7f144c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c004810 0x7f144c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.502 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 -- 172.21.15.98:0/2406433088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c003570 con 0x7f144c005210 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 -- 172.21.15.98:0/2406433088 reap_dead start 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146694e700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f144c00ba60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f145800a960 tx=0x7f14580021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e78453cba335013 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580070d0 con 0x7f144c005210 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.494+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1458007230 con 0x7f144c005210 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f146514b700 1 -- 172.21.15.98:0/2406433088 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580075f0 con 0x7f144c005210 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 -- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 msgr2=0x7f144c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f144c00ba60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f145800a960 tx=0x7f14580021d0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 -- 172.21.15.98:0/2406433088 shutdown_connections 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 --2- 172.21.15.98:0/2406433088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f144c00ba60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 -- 172.21.15.98:0/2406433088 >> 172.21.15.98:0/2406433088 conn(0x7f1460099830 msgr2=0x7f146006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:07.503 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 -- 172.21.15.98:0/2406433088 shutdown_connections 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.495+0000 7f14683b1700 1 -- 172.21.15.98:0/2406433088 wait complete. 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.496+0000 7f14683b1700 1 Processor -- start 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.496+0000 7f14683b1700 1 -- start start 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 0x7f1460157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 0x7f146009b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146016dff0 con 0x7f144c005210 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146009add0 con 0x7f146009b260 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f14683b1700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146009a1c0 con 0x7f1460158530 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41396/0 (socket says 172.21.15.98:41396) 2023-10-06T01:42:07.504 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 -- 172.21.15.98:0/686092999 learned_addr learned my addr 172.21.15.98:0/686092999 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146694e700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 0x7f146009b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146594c700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 0x7f1460157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 -- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 msgr2=0x7f1460157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 0x7f1460157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 -- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 msgr2=0x7f146009b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 0x7f146009b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 -- 172.21.15.98:0/686092999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146009a4a0 con 0x7f144c005210 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146594c700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 0x7f1460157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146694e700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 0x7f146009b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.497+0000 7f146614d700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f145c002970 tx=0x7f145c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00f8d0 con 0x7f144c005210 2023-10-06T01:42:07.505 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f145c00fa30 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00fdb0 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146009a780 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1460158250 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.498+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1460064ad0 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.499+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f145c013600 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.499+0000 7f14567fc700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 0x7f1444069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.499+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f145c084a80 con 0x7f144c005210 2023-10-06T01:42:07.506 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.499+0000 7f146594c700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 0x7f1444069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T01:42:07.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.500+0000 7f146594c700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 0x7f1444069490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1450006eb0 tx=0x7f1450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T01:42:07.507 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.503+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f145c051cd0 con 0x7f144c005210 2023-10-06T01:42:07.729 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.729+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14600b1820 con 0x7f144c005210 2023-10-06T01:42:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.729+0000 7f14567fc700 1 -- 172.21.15.98:0/686092999 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f145c0552f0 con 0x7f144c005210 2023-10-06T01:42:07.730 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-10-06T01:42:07.730 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5161,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2800041409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2800041409},{"type":"v1","addr":"172.21.15.184:6834","nonce":2800041409}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/998571123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":998571123},{"type":"v1","addr":"172.21.15.184:6835","nonce":998571123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3383350652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3383350652},{"type":"v1","addr":"172.21.15.98:6835","nonce":3383350652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4176896243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4176896243},{"type":"v1","addr":"172.21.15.98:6837","nonce":4176896243}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[]} 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 msgr2=0x7f1444069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 0x7f1444069490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1450006eb0 tx=0x7f1450008040 comp rx=0 tx=0).stop 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 msgr2=0x7f146016dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f145c002970 tx=0x7f145c007d80 comp rx=0 tx=0).stop 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 shutdown_connections 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:6832/120423,v1:172.21.15.98:6833/120423] conn(0x7f1444066fe0 0x7f1444069490 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146009b260 0x7f146009b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c005210 0x7f146016dab0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 --2- 172.21.15.98:0/686092999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1460158530 0x7f1460157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T01:42:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 >> 172.21.15.98:0/686092999 conn(0x7f1460099830 msgr2=0x7f14600b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T01:42:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 shutdown_connections 2023-10-06T01:42:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2023-10-06T01:42:07.731+0000 7f14683b1700 1 -- 172.21.15.98:0/686092999 wait complete. 2023-10-06T01:42:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 31 2023-10-06T01:42:07.743 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T01:42:07.744 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T01:42:07.744 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T01:42:07.744 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.752 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T01:42:07.753 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T01:42:07.753 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.802 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T01:42:07.802 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T01:42:07.802 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.813 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T01:42:07.813 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T01:42:07.813 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.864 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T01:42:07.864 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T01:42:07.864 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T01:42:07.864 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.889 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T01:42:07.889 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T01:42:07.890 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:07.954 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T01:42:07.954 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T01:42:07.954 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.019 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T01:42:08.019 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T01:42:08.019 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.085 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T01:42:08.086 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T01:42:08.086 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.124 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T01:42:08.124 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T01:42:08.124 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.206 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T01:42:08.206 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T01:42:08.206 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.274 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T01:42:08.274 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T01:42:08.274 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.337 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T01:42:08.338 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T01:42:08.338 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T01:42:08.338 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.362 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T01:42:08.363 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T01:42:08.363 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.401 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T01:42:08.401 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T01:42:08.402 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T01:42:08.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.413 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T01:42:08.413 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T01:42:08.413 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.453 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T01:42:08.453 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T01:42:08.453 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T01:42:08.503 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T01:42:08.503 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T01:42:08.504 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T01:42:08.533 INFO:teuthology.orchestra.run.smithi098.stdout:1696556384.3029299 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T01:42:08.533 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T01:42:08.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T01:42:08.603 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T01:42:08.673 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T01:42:08.742 INFO:teuthology.orchestra.run.smithi098.stdout:1696556384.3029299 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T01:42:08.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:42:08.743 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T01:42:08.947 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:42:08.947 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T01:42:09.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:42:09.056 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T01:42:09.173 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:42:09.173 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T01:42:09.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:42:09.237 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T01:42:09.440 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:42:09.440 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T01:42:09.540 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:42:09.540 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T01:42:09.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:42:09.616 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T01:42:09.684 INFO:tasks.ceph:Archiving mon data... 2023-10-06T01:42:09.685 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/data/mon.a.tgz 2023-10-06T01:42:09.685 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2023-10-06T01:42:09.700 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.sahueAvfX0 2023-10-06T01:42:09.700 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sahueAvfX0 2023-10-06T01:42:09.826 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.sahueAvfX0 2023-10-06T01:42:09.906 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.sahueAvfX0 is 326KB 2023-10-06T01:42:10.001 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.sahueAvfX0 2023-10-06T01:42:10.015 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/data/mon.b.tgz 2023-10-06T01:42:10.015 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-06T01:42:10.031 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.3h6k0an5e1 2023-10-06T01:42:10.031 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3h6k0an5e1 2023-10-06T01:42:10.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.3h6k0an5e1 2023-10-06T01:42:10.273 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.3h6k0an5e1 is 433KB 2023-10-06T01:42:10.370 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.3h6k0an5e1 2023-10-06T01:42:10.383 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/data/mon.c.tgz 2023-10-06T01:42:10.384 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-06T01:42:10.436 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.0Es3n5LTc2 2023-10-06T01:42:10.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0Es3n5LTc2 2023-10-06T01:42:10.580 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.0Es3n5LTc2 2023-10-06T01:42:10.661 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.0Es3n5LTc2 is 433KB 2023-10-06T01:42:10.757 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.0Es3n5LTc2 2023-10-06T01:42:10.771 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T01:42:10.772 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T01:42:10.774 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T01:42:10.834 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T01:42:10.835 DEBUG:teuthology.orchestra.run.smithi098:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T01:42:10.840 DEBUG:teuthology.orchestra.run.smithi184:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T01:42:10.865 INFO:teuthology.orchestra.run.smithi098.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T01:42:10.900 INFO:teuthology.orchestra.run.smithi184.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T01:42:10.906 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T01:42:10.907 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi098/crash 2023-10-06T01:42:10.908 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T01:42:10.936 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi184/crash 2023-10-06T01:42:10.937 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T01:42:10.969 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T01:42:10.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:42:10.981 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T01:42:11.057 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T01:42:11.080 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T01:42:11.104 INFO:tasks.ceph:Compressing logs... 2023-10-06T01:42:11.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T01:42:11.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T01:42:14.637 INFO:tasks.ceph:Archiving logs... 2023-10-06T01:42:14.638 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi098/log 2023-10-06T01:42:14.639 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T01:42:15.275 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi184/log 2023-10-06T01:42:15.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T01:42:15.687 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T01:42:15.696 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T01:42:15.697 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T01:42:15.700 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T01:42:15.785 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T01:42:15.795 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T01:42:15.795 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T01:42:15.797 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T01:42:15.809 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-10-06T01:42:15.811 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:210 Number of sources = 4 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +34us[ +34us] +/- 77ms 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -436us[ -441us] +/- 64ms 2023-10-06T01:42:15.812 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +422us[ +422us] +/- 72ms 2023-10-06T01:42:15.813 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:210 Number of sources = 4 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +35us[ +35us] +/- 77ms 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -429us[ -431us] +/- 64ms 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +440us[ +439us] +/- 72ms 2023-10-06T01:42:15.814 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T01:42:15.815 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T01:42:15.825 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T01:42:15.826 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T01:42:15.839 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T01:42:15.950 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T01:42:16.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T01:42:16.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:42:16.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T01:42:16.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:42:16.101 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T01:42:16.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T01:42:16.133 INFO:teuthology.task.internal:Duration was 1386.692626 seconds 2023-10-06T01:42:16.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T01:42:16.149 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T01:42:16.149 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T01:42:16.152 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T01:42:16.193 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T01:42:16.196 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T01:42:16.431 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T01:42:16.431 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2023-10-06T01:42:16.432 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T01:42:16.490 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2023-10-06T01:42:16.491 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T01:42:16.509 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T01:42:16.510 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T01:42:16.532 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T01:42:16.588 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T01:42:16.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T01:42:16.612 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T01:42:16.700 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T01:42:16.710 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T01:42:16.710 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T01:42:16.734 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T01:42:16.759 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T01:42:16.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T01:42:16.777 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T01:42:16.798 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2023-10-06T01:42:16.822 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2023-10-06T01:42:16.836 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T01:42:16.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:42:16.865 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T01:42:16.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T01:42:16.890 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T01:42:16.901 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T01:42:16.901 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi098 2023-10-06T01:42:16.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T01:42:16.941 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088/remote/smithi184 2023-10-06T01:42:16.941 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T01:42:16.975 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T01:42:16.975 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T01:42:16.977 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T01:42:17.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T01:42:17.034 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T01:42:17.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T01:42:17.044 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T01:42:17.044 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T01:42:17.047 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T01:42:17.058 INFO:teuthology.orchestra.run.smithi098.stdout: 262184 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 01:42 /home/ubuntu/cephtest 2023-10-06T01:42:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: 262176 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 01:42 /home/ubuntu/cephtest 2023-10-06T01:42:17.078 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T01:42:17.088 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T01:42:17.109 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T01:42:17.134 DEBUG:teuthology.nuke:shortname: smithi098 2023-10-06T01:42:17.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T01:42:17.158 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:12:01.034911', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPjxjWApRZqgOMmRkqAKtxkRzr0GOGEOY3Hf3J2iWiMgMQJMtCLfNCMwNhkeTMl99ekKWHz1qTVHUFzssA54Bc='} 2023-10-06T01:42:17.161 DEBUG:teuthology.nuke:shortname: smithi184 2023-10-06T01:42:17.162 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T01:42:17.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412088', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 01:12:01.036913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI00sWvHkKzqXh1e1/v1VTW2/GSd8VRipyNfxlH5V5o99+A1owTp6qbpism66RZ736kOpXoRoX8tHFblaGu4KLs='} 2023-10-06T01:42:17.209 INFO:teuthology.orchestra.console.smithi098:Power off 2023-10-06T01:42:17.209 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T01:42:17.232 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-10-06T01:42:17.232 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:17.247 INFO:teuthology.orchestra.console.smithi184:Power off 2023-10-06T01:42:17.247 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T01:42:17.256 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-10-06T01:42:17.269 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2023-10-06T01:42:17.269 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:17.294 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-06T01:42:21.257 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:21.295 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:21.367 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-10-06T01:42:21.406 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-06T01:42:25.370 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:25.407 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:25.479 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-10-06T01:42:25.514 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-06T01:42:25.581 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-10-06T01:42:29.517 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T01:42:29.626 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2023-10-06T01:42:29.727 INFO:teuthology.orchestra.console.smithi184:Power off completed 2023-10-06T01:42:29.832 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} duration: 1386.6926262378693 failure_reason: '"1696556384.3029299 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T01:42:29.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T01:42:29.916 INFO:teuthology.run:FAIL